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ABSTRACT 



In 1996, the Chicago Public Schools (CPS) began an 
initiative aimed at ending social promotion and raising achievement. The 
centerpiece of the initiative is a set of test-score cutoffs for third, 
sixth, and eighth graders, who must achieve a minimum score on standardized 
reading and mathematics tests to be promoted to the next grade. Students who 
do not meet the criteria are required to attend a special summer school 
program. Summer Bridge. Those who fail again are retained in their grade or, 
if they are 15, are sent to new alternative schools called Transition 
Centers. This report describes results from the first 2 years of the 
initiative and identifies many important issues that merit further study. 
Compared are the performance of students subject to the policy in 1997 and 
1998 with that of a previous group of CPS students not subject to the 
criteria. There have been impressive increases in the number of students who 
meet the minimum test score cutoffs for promotion. The performance of 
students with low skills shows the greatest improvement, but the picture is 
mixed on whether getting students up to a test -score cutoff in one year 
allows them to do better the next year. Large test score increases in Summer 
Bridge were not followed by improved performance the next year. Results from 
the first group of retained students are also not encouraging. Chicago has 
not solved the problem of poor performance among those who did not meet the 
minimum score cutoffs. Research steps to examine the effects of the program 
further are outlined. An appendix contains some detailed test results for 
first- time test takers. (Contains 25 figures, 38 endnotes, and 26 
references.) (SLD) 



Reproductions supplied by EDRS are the best that can be made 
from the original document. 
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Pictured is Keith Collins, then a student at William E. Gladstone Elementary School. Keith currently 
attends John M. Smyth Elementary School. 
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n 1996, the Chicago Public Schools (CPS) began an ambitious new 
initiative aimed at ending social promotion and raising achievement.^ 
The centerpiece of this initiative is a set of promotional test-score cut- 
offs for third, sixth, and eighth graders. Students in these grades must 
achieve a minimum score on a standardized test in reading and mathemat- 
ics in order to be promoted to the next grade. Students who do not meet 
the criteria are required to participate in a special summer school program. 
Summer Bridge, and retake the test at the end of the summer. Those who 
fail again are retained in their grade or, if they are 15, are sent to new 
alternative schools called Transition Centers. In the first two years under 
the policy, more than one-third of third, sixth, and eighth graders failed to 
meet the promotional test cutoffs by the end of the school year. Of these, 
more than 22,000 students attended Summer Bridge. At the end of the 
summer, 10,000 of them met the test criteria and were promoted. In both 
1997 and 1998, CPS retained 20 percent of eligible third graders and 
approximately 10 percent of sixth and eighth grade students. In 1998, 
almost 1,600 students were retained for a second time. 

It is not an overstatement to say that all eyes are on Chicago. CPS s 
efforts have spurred a wave of similar reforms in school systems around the 
country, and the “hazards of social promotion” have become a mantra in 
political speeches. President Clinton heralded this initiative in his 1999 
State of the Union Address, arguing: 



When we promote a child from grade to grade who hasnt mastered the 
work, we do that child no favors. It is time to end social promotion in 
Americas schools. Last year in Chicago, they made that decision. . . .1 
propose to help other communities follow Chicago's lead. 



Many educators criticize Chicago’s policy, however, for focusing on 
simplistic solutions and particularly for relying on a practice — grade 
retention — that has not been shown to lead to higher achievement. Rob- 
ert Hauser, chair of a National Research Council panel on the appropriate 
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use of testing, argued passionately that the prepon- 
derance of evidence shows negative consequences of 
retaining students: 

should know that a new policy works before we 
try it out on a large scale. In its plan to end social 
promotion, the [national] administration appears 
to have mixed a number of fine proposals for edu~ 
cational reform with an enforcement provision — 
flunking kids by the carload lot — about which 
the great mass of evidence is strongly negative. 
And this policy will hurt poor and minority chil' 
dren most of all. ^ 

In addition, the Chicago policy is criticized because 
the practice of making promotional decisions based 
on a one-time test score is inconsistent with profes- 
sional standards.^ (See Sidebar on page 5.) A panel of 
the National Research Council recently came out 
strongly against the sole use of test scores for making 
promotional decisions, taking the stand that high stakes 
testing should occur only after instructional changes 
have been made/ 

Given the rhetoric and attention surrounding this 
initiative, it is critical that public debate be informed 
by an understanding of what the Chicago policy actu- 
ally is and the best available evidence of its effects on 
student achievement, student progress, and on instruc- 
tion. This is the first in a series of reports the Consor- 
tium on Chicago School Research will produce over 
the next several years as part of a larger multi-year study 
of the effect of Chicago's promotion policies on stu- 
dents’ opportunities to learn and on their long-range 
school outcomes. This first report describes the imple- 
mentation of the policy during the first two years and 
the major processes at work. It tracks the flows of stu- 
dents through the policy, compares the progress of stu- 
dents who faced the promotional test cutoffs in 1997 
and 1998 with that of a group of students before the 
policy, and examines how students’ experiences vary 
by race and gender. Subsequent reports will evaluate 
more specifically the nature of achievement effects as- 



sociated with the policy for different groups of stu- 
dents over time. 

In the process of describing results from the first 
two years, this report identifies many important ques- 
tions and areas of concern that merit future investiga- 
tion. We have already begun work on some of these. 
An important purpose in releasing this first report is 
to stimulate further public conversation about these 
efforts. In so doing, we expect to identify more im- 
portant questions that will help shape our continuing 
research agenda on this important policy initiative. 

ATheoiy of Action: What Is Chicago's 
Effort to End Social Promotion? 

T he CPS policy was enacted to address two concerns: 
First, students were having difficulty in later grades, 
particularly in high school, because they had been al- 
lowed to progress through elementary school without 
attaining even minimum levels of basic skills. The sec- 
ond concern was raised by teachers: How could they 
pursue higher standards or be accountable for poor 
student performance if students did not have the skills 
to move on to more advanced material? 

The CPS initiative aims to address these problems 
through a combination of efforts — during the testing 
year, over the summer, and during the retention year — 
designed to raise students’ skills to meet minimum test 
scores on the Iowa Tests of Basic Skills (ITBS) before 
they are promoted. First, in the year before promo- 
tion, the policy seeks to use the threat of retention as 
an incentive to motivate students to work harder and 
to encourage parents to monitor their children’s per- 
formance more closely. The policy aims to focus teacher 
attention on those students who are not mastering the 
material and send a strong message to cover material 
that will raise students’ skills. In addition, students who 
are at risk are given extended instructional time dur- 
ing the school year through Lighthouse, an after-school 
program that began in 1997 and was expanded in 
1998. Lighthouse provides schools with funds to ex- 
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tend the school day and a centrally developed curricu- 
lum focused on reading and mathematics. 

The second major component of the policy is the 
Summer Bridge program, which provides additional, 
more focused, instructional time and a second chance 
to pass the test cutoff during the summer. This much 
heralded program offers smaller classes and a centrally 
mandated curriculum aligned with the format and 
content of the ITBS. 

And, third, the policy uses the practice of grade re- 
tention and directs even more resources in the retained 
year in an effort to get students back on track. Schools 
with high proportions of retained students have been 
given extra teachers and reduced class sizes. Retained 
students are also required to participate in the Light- 
house after-school program. In addition, CPS is ex- 
perimenting with a range of additional policy strategies, 
including retesting mid-year (January) so that retained 
students who then pass the test cutoff can rejoin their 
classmates.^ In total, the policy combines high stakes 
testing with multiple chances to reach the minimum 
ITBS score and progressively targeted intervention, all 
aimed at improving the achievement of students with 
the lowest skills. 




Proponents of such initiatives argue that raising stu- 
dents' basic skills before they are allowed to move on 
to the next grade is essential for long-term school suc- 
cess. While low-achieving students should benefit the 
most, all students will benefit because they will receive 
more focused instruction and will be in classrooms 
where students are working harder and are on task. 
The policy also seeks to address educators’ concerns 
that social promotion hampers the ability to teach 
grade-appropriate material. By ensuring that students 
have the prerequisite basic skills to tackle more chal- 
lenging material, the policy attempts to allay this fear. 
Thus, all students should benefit because their teach- 
ers in later grades will be able to pursue more advanced 
objectives and use more grade-appropriate content. 



Critics of the policy worry about three potential 
negative effects. First, critics worry that the policy en- 
courages too great a focus on test preparation and ba- 
sic skills drills and leads teachers to limit content 
coverage, slowing down rather than increasing the pace 
of instruction in the testing years. Second, critics ar- 
gue that the practice of retaining students has not been 
shown to produce increases in achievement, even with 
remediation.^ They also note that research evidence 
suggests that retention has long-term negative effects 
on students’ self-esteem and school attachment and is 
associated with higher dropout rates. ^ Thus, retention 
and the placement of students in Transition Centers 
may benefit those who are promoted, while creating 
sacrificial lambs of the most vulnerable Chicago stu- 
dents. And third, critics of the policy worry that link- 
ing decisions to a single test score creates pressure that 
might result in cheating or might lead well-intentioned 
educators to try to protect students who are at risk by 
placing them in special education or retaining them 
earlier. Many teachers believe that retaining students 
in earlier grades is better than retention in later 
grades, but this practice has not been shown to have 
positive results.^ 

Previous policy initiatives similar to Chicago’s have 
not had a successful track record in this respect. In the 
early 1980s, New York City engaged in a similar ef- 
fort, giving students who did not meet a “promotional 
gate” extra summer resources and reduced class sizes. 
In an evaluation of the New York initiative, Ernest 
House found that students who had been retained 
under the policy had similar test scores in post-pro- 
motional gate grades to a matched group of low-per- 
forming students who had been socially promoted 
before the policy.^ He concluded that retention and 
extra resources provided no benefit to these students. 
House found, moreover, that retained students 
dropped out at significantly higher rates (40 versus 25 
percent) than the matched group of previously pro- 
moted, low-achieving students. 

In taking on social promotion, Chicago is attempt- 
ing to confront one of the most persistent problems in 
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education. How can we address consistently poor per- 
formance among urban students and in urban schools? 
On the one hand, sending students with low skills into 
high school and into the labor market sets them up 
for failure. On the other hand, the most commonly 
employed alternative, grade retention, may be as prob- 
lematic or even worse. All of this suggests that Chi- 
cago is facing a tall order in using the threat of retention 
as a means to motivate students and teachers, while at 
the same time using retention itself as a means to 
remediate poor performance. 



Qiiragos eppreacli is mis siiplia policy 
bit more of m ioleiroled set of ioitio- 
tives focisiii elleiitioDi oo ie poorest 
performiii|sti]deiits... 



Unfortunately, there is little research to support or 
negate the central premise of the promotional initia- 
tive in Chicago — that setting minimum test-score cut- 
offs for students will lead to more focused instruction 
and higher achievement, and will lay the basis for long- 
term school success.^® Prior studies have focused al- 
most exclusively on the impact of retention. We know 
little about whether the introduction of high stakes 
testing, with linked support efforts such as Summer 
Bridge and Lighthouse, will affect greater learning gains 
for students who are promoted. Nor do we know 
whether reducing the spread of achievement in post- 
promotional grades will lead teachers to pursue more 
difficult content and skills coverage. 

Past research clearly supports the CPS policy in one 
area— greater instructional time has positive effects, 
particularly when it is positioned during the summer. 



Multiple studies document that impoverished students 
lose ground during the summer months and that this 
“summer learning loss” may be an important reason 
why poor children fall behind their more advantaged 
counterparts.^^ 

The Current Study 

Ending social promotion is a much more complex 
undertaking than might at first be imagined. Chicago’s 
approach is not a single policy but more of an inte- 
grated set of initiatives focusing attention on the poor- 
est performing students during the school year before 
testing, over the summer, and in the year after reten- 
tion. Clearly, some components of the policy may work 
more effectively than others for different groups of stu- 
dents. They also require varying levels of resources. 
This report and those that follow will focus in more 
detail on untangling the web of effects associated with 
each of the components of this initiative. 

We will also be looking at changes in the policy 
over time and evaluating how such changes shape the 
policy’s impact. It is important to recognize that CPS’s 
policy has been evolving. For example, the adminis- 
tration argues that the sole use of test scores in the 
first years of the policy was intended to set a “gold 
standard” in a school system where grades had lost 
their meaning as indicators of student knowledge. 
Three years after implementation, Chicago has decided 
to raise the minimum test score needed for promo- 
tion in all three grades. At the same time, the CPS 
administration has stated that the criteria for promo- 
tion will be expanded to include grades, attendance, 
and learning growth over the school year. Similarly, 
the administration has added new program compo- 
nents, such as expanding Summer Bridge to first and 
second graders who have ITBS scores below grade level. 
Existing components may also be modified over time. 
Clearly, tracking the implementation and effect of these 
changes will be an important focus of future work. 
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Major professional organizations concerned with testing, in- 
cluding the American Psychological Association, the National 
Council on Measurement in Education, and the American 
Educational Research Association, have all taken stands op- 
posing the use of a single test score in making promotional 
decisions. Test publishers note in their technical documen- 
tation that it is inappropriate to use test scores, taken alone, 
for deciding whether to retain students. A recent report 
of the National Research Council on high stakes testing 
concluded: 

Scores from large-scale assessments should never be the 
only sources of information used to make a promotion or 
retention decision. No single source of information — 
whether test scoresy course grades y or teacher judgments — 
should stand alone in making promotion decisions. Test 
scores should always be used in combination with other 
sources of information about student achievement. * 

t 

The Consortiums own work confirms these conclusions. 
In the fall of 1998, the Consortium released a major report 
on Iowa Tests of Basic Skills (ITBS) score trends in Chicago 
and the increased use of these data in a new high stakes ac- 
countability environment.^ Many of the issues raised in this 
report have implications for the current promotional policy, 
which requires reaching particular test scores for promotion 
to the next grade. 

Why do experts take such a strong stand against using 
single test scores? The reason is that testing is an imprecise 
science. There are two forms of imprecision on the ITBS: 
differences in content and difficulty from form to form and 
numerous distinctions that are being made based on a small 
amount of information. 

First, CPS sets a minimum test score in Grade Equiva- 
lents (GEs) for promotion at grades three, six, and eight. 
The system currently employs several different forms of the 
ITBS, which it administers at different times. Since each form 
and level of the test produces GEs, one might easily think 
that these results are equivalent and directly comparable. In 
fact, the Consortiums study showed that they are not. Rather, 
the Grade Equivalent metric is form- and level-specific; con- 
sequently, results are not strictly comparable across different 
forms and levels. This is not a problem for the purpose for 
which the test was originally intended — to get a quick com- 
parison of student performance relative to a national sample 



that took the exact same test. It is a problem, however, when 
we seek to establish a GE score as a minimum standard. Since 
different forms of the ITBS are administered from year to 
year (and each has a somewhat different set of GEs), stu- 
dents actually confront varying degrees of risk of failure de- 
pending upon the particular test form used that year. 

Second, test scores are imprecise because there are only a 
set number of questions on a test and many possible GE 
cutoffs. For example, in Form M, used in both 1997 and 
1999, there are only 48 reading questions on the eighth- 
grade test but the GE range spans from a low of 1 .9 to a high 
of 16.3. If all of the Grade Equivalents in this range were 
possible, the test would be making 134 distinctions on the 
basis of 48 questions — clearly an impossibility. As a result, 
there are many test scores that are simply impossible to ob- 
tain on the ITBS. On Form M, for example, a student can 
either receive a 6.9 or, if they got one more item correct, a 
7.3. Scores from 7.0 to 7.2 don’t exist on the eighth grade 
Form M reading test. It is not unusual that getting just one 
more item correct can make a difference of .3 to .4 GEs. 

Test makers take these problems into account by using a 
concept called the standard error of measurement. The stan- 
dard error of measurement associated with an individual’s 
test score tells us how precise the individual score report ac- 
tually is. For the upper grades on the ITBS, the standard 
errors of measurement in GEs are quite large. For example, 
the CPS established a cut-score of 7.4 GEs for graduation 
from eighth grade. The standard error of measurement, based 
on the Level 14 test used in 1998, is almost 0.9 GEs for a 
student who is at national norms (i.e., 8.8 GEs for an eighth 
grader). This means that it is quite plausible for this student 
to produce a test score that ranges anywhere from 7.0, fall- 
ing below the cutoff for promotion, to 10.6 GEs, almost 
two years above grade level. (Formally, this is called a 95 
percent probability interval — a range of two standard errors 
in either direction of a particular score).^ 

Finally, the ITBS is not aligned with either the Chicago 
Academic Standards or the Illinois Learning Standards. As a 
result, the specific competencies required for promotion are 
not publicly stated. 



‘Heubert and Hauser (1999), p. 286. 

^Bryk, Thum, Easton, and Luppescu (1998). 
^See Rogosa (1999). 
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W hat does it mean to end social promotion in a school system the 
size of Chicago’s? This section looks at the aggregate statistics for 
the first two years of the policy regarding how many students 
were retained, promoted, or met the minimum test-score cutoff for pro- 
motion. These numbers reflect a series of important outcomes at each stage 
of the process — the effectiveness of efforts to raise test scores during the 
school year, the effect of the second chance in Summer Bridge, and the 
effect of efforts in the following year to address poor performance among 
retained students. In addition, these statistics reflect the impact of admin- 
istrative decisions about which students are included in the policy and 
whether students who do not meet the cutoff are promoted anyway. 

Results for the first Yeai 1 996-1 997 

Who Was Subject to the Policy? 

Main finding: Thirty-one percent of third graders and 20 percent of sixth and 
eighth graders were not subject to the policy. For third graders, participation in 
bilingual education was the primary reason for exclusion. For sixth and eighth 
graders, classification in special education programs was the primary reason for 
exclusion. 

The first decision a school system faces in trying to end social promotion 
is determining which students to include in the policy. The CPS decided 
to focus its efforts in the third, sixth, and eighth grades. In prior years, 
these were the grades for a state-administered test, the Illinois Goal Assessment 
Program (IGAP). In 1997, CPS decided that the promotional decisions for 
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Figure 1*1 

Summary Table; Students Excluded from 
the Test Score Cutoff in May 1997 







Reason for Exclusion among Those Tested 




Percent of 
first time 
students 
In grade 
excluded ^ 


Excluded 
because 
not tested^ 


Special 

education 


Bilingual 


Special 

education 

and 

bilingual 


Unknown 


Total 

tested but 
excluded 


Total 

excluded 


Third 


6,631 


2,100 


1,604 


248 


49 


4,001 


10,632 


31% 


Sixth 


2,012 


3,233 


381 


394 


40 


4,048 


6,060 


19% 


Eighth 


2,112 


3,156 


354 


325 


33 


3,868 


5,980 


21% 



Students "excluded but not tested" did not take the ITBS because of their bilingual or special education status or 
were tested in Spanish. Most third graders not tested were in bilingual classes. Other excluded students took the 
ITBS, but fell under one of the two exclusion criteria. 



First time students are those who spend one year in third, sixth, or eighth grade, thus excluding retained students. 



two groups of students in these grades would not be 
made solely on the basis of scores on the Iowa Tests of 
Basic Skills (ITBS) — students who were in bilingual 
education fewer than three years and students who were 
in graded special education classrooms/^ Some of these 
students were not tested — largely because of their lim- 
ited proficiency in English — while others were tested 
but fell under one of the exclusion criteria. In this re* 
port, we call both students who weren’t tested and those 
whose tests were excluded from the policy excluded 
students (see Figure 1-1). We call students in these 
grades included if they were tested and their pro- 
motional decisions were made on the basis of their 
ITBS scores. 



See the Consortium’s web page for an 
executive summary of this report: 
M4[p“//www.coDisoirtiym=clhiicago.oirg 



ERIC 



Figures 1-2, 1-3, and 1-4 (on pages 10-15) show 
the numbers of first time third, sixth, and eighth grad- 
ers who were excluded in 1997. The decision to ex- 
clude students who were in bilingual classrooms for 
fewer than three years meant that many third graders 
were excluded for that reason. As students move 
through grades, special education placements rise, and 
the proportion of students who have been in bilingual 
education for fewer than three years falls. This meant 
that about 80 percent of sixth and eighth graders were 
included under the policy, whereas less than 70 per- 
cent of third graders were included. As seen in Figure 
1-1, among those students who were excluded, most 
sixth and eighth graders were excluded because of 
their special education status. (Section 1 continues 
on page 16.) 
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Sidebar 2 



One of the reasons that third graders had lower passing rates is that third graders were much more likely to be behind 
in both reading and mathematics. Over half of third graders who failed to meet the minimum cutoff in May 1997 
were below 2.8 in both reading and mathematics. Students who failed in both subjects had a hard time bringing their 
scores up in Summer Bridge. Less than 20 percent of students who had to attend Summer Bridge to raise their test 
scores in both reading and mathematics managed to meet the test cutoff in both subjects by the end of the summer, 
regardless of whether they were third, sixth, or eighth graders. In comparison, about half of students who needed to 
raise their test scores in only one subject managed to accomplish that by the end of the summer. 

Students who failed in both subjects differed in several respects. First, these students started farther behind. The 
average third grade reading score for students who failed only reading was 2.22 in May 1997, compared to 1.84 for 
third graders who failed both subjects. Similar differences were also observed for sixth and eighth graders who failed 
both subjects. 

Second, many more of these students were not tested at the end of the summer, suggesting that they had not 
participated in Summer Bridge. Almost 20 percent of third, sixth, and eighth grade students who failed to meet the 
test cutoff in both subjects in 



Reasons for Not Meeting the Cutoff for Promotion 
in May 1997, and End of Summer Results 



May were not retested in August 
compared to 1 0 percent of third 
graders and 13 percent of sixth 
and eighth graders who failed in 
reading only. 

And, finally, this group of 
students who missed the cutoff 
in both subjects had smaller 
testing gains in Summer Bridge 
than did students who failed 
only one subject. Smaller test- 
ing gains in Summer Bridge 
among this group may reflect 
both the fact that they started 
farther behind and that trying 
to pass two subjects in one 
summer is difficult. In addi- 
tion, these lower passing rates 
may signify motivational or 
other difficulties. 

A better understanding of 
the characteristics of this group 
of students is important, as they 
make up the majority of those 
who are retained. Indeed, 71 
percent of third graders who 
were retained came from the 

group of students who failed to reach the test cutoff in both reading and mathematics in May. We will be examining 
these issues more closely in further research. 





Failed 

Reading^ 


Failed 

Math 


Failed both 
Reading 
and Math 


Total 




% 

total 




% 

total 




% 

total 




Grade 3 
















Did not meet cutoff in May 


3,988 


36% 


1,527 


14% 


5,650 


51% 


11,165 


Did not meet cutoff by August 


2,091 




423 




4,677 




7,191 


Proportion did not meet cutoff by August 


52% 




28% 




83% 




64% 


Retained/transition center 


1,195 


25% 


190 


4% 


3,411 


71% 


4,796 


Grade 6 
















Did not meet cutoff in May 


4,219 


50% 


1,234 


15% 


2,979 


35% 


8,432 


Did not meet cutoff by August 


1,857 




591 




2,489 




4,937 


Proportion did not meet cutoff by August 


44% 




48% 




83% 




58% 


Retained/transition Center 


1,099 


36% 


311 


10% 


1,629 


53% 


3,039 


Grade 8 
















Did not meet cutoff in May 


2,688 


46% 


1,192 


20% 


1,959 


34% 


5,819 


Did not meet cutoff by August 


1,324 




611 




1,582 




3,517 


Proportion did not meet cutoff by August 


50% 




51% 




81% 




61% 


Retained/transition center 


863 


37% 


243 


11% 


1,182 


52% 


2,288 



"Failed" means that students' test score in that subject did not meet the promotional 
cutoff-2.8 for third graders, 5.3 for sixth graders, and 7.0 for eighth graders. 



o 

ERIC 



It 



10 



Figure 1-2 



1997-1 998 Test and Retention Results 

All 3rd Graders in Spring 1997 



Spring 1997 Summer 1997 Fall 1997 

Test results Test results Actions 



Fall 1997 
Summary 
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1997-1 998 Test and Retention Results 

All 3rd Graders in Spring 1997 

Details 



1 1 



1 The excluded category includes students who took the 
test, but whose scores were excluded from reporting, and 
students who did not take the test. Of those students who 
took the test, the test scores of 4,001 students were ex- 
cluded from reporting. Of these, 1,600 were bilingual and 
2,149 were special education students. An additional 6,631 
students did not take the test, perhaps because of special 
education or bilingual status. 



2 3,872 of the 4,101 students who passed the Summer 
Bridge ITBS were promoted, 1 02 were retained for reasons 
other than failing the test (e.g., poor attendance), and 127 
left the system. 



3 1,855 of the 5,648 students who failed Summer 
Bridge were waived and promoted, 3,602 were re- 
tained, 189 left the system, and 2 moved into non- 
graded special education. 



4 642 of the 2,002 students who did not take the test 
in Summer Bridge were promoted, 1 ,092 were retained, 
266 left the system, and 2 moved into non-graded spe- 
cial education. 



5 The numbers in this column do not add up to exactly 34,775 
because 78 students moved into non-graded special edu- 
cation between the two semesters. 



6 The numbers in this column do not add up to exactly 5,551 
for the following reasons: 1 ) 205 of the retained 3rd graders 
were enrolled in the fall semester, but were not enrolled in 
the system the following spring semester when the test was 



re-taken; 2) 189 of the retained 3rd graders were promoted 
to 4th grade between the fall and spring semesters; 3) 7 of 
the students moved into non-graded special education be- 
tween the fall and spring semesters; and 4) 45 of the stu- 
dents were reclassified into a grade other than 3rd or 4th 
between the fall and spring semesters - demoted to 1st or 
2nd grade, or promoted to 5th or 6th grade. In some cases 
this was most likely a recording error in the school records. 
The numbers in this figure trace those 5,105 students that 
were classified as repeat 3rd graders in fall 1998. 



7 The test scores of 715 students were excluded from re- 
porting. Of these, 162 were bilingual, 512 were special edu- 
cation, and 41 were both bilingual and special education 
students. An additional 261 students did not take the test, 
perhaps because of special education or bilingual status. 



8 573 of the 61 7 students who passed Summer Bridge 1 998 
were promoted, 26 were retained, and 18 left the system. 



9 365 of the 1 ,1 69 students who failed summer Bridge were 
promoted, 754 were retained, 49 left the system, and 1 was 
reclassified as non-graded special education. 



10 276 of the 579 students who did not take the test at 
Summer Bridge were promoted, 249 were retained, 48 
left the system, and 6 were reclassified as non-graded 
special education. 



1 1 The numbers in this column do not add up to exactly 5,551 
for the reasons listed in Footnote 6, and because 17 stu- 
dents moved into non-graded special education between 
the two semesters. 



Retained 3rd Graders in Spring 1998 

Spring 1998 Summer 1998 Fall 1998 

Test results results Actions 



Fall 1998 
Summary 



1,764 

Passed 










1,685 

Promoted 


2,365 

Failed 






617 Passed® 

9 






57S^d ' ^ 
641 Did not pass 

' ' 

832 Promoted 






1,169 Failed 
579 No test 






976 Excluded^ 
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5,551® 5,551^^ 
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Figure 1-3 



1 997-1 998 Test and Retention Results 

All 6th Graders in Spring 1997 



Spring 1997 Summer 1997 Fall 1997 

Test results Test results Actions 



Fall 1997 
Summary 
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1 997*1 998 Test and Retention Results 

All 6th Graders in Spring 1997 

Details 



13 



1 The excluded category includes students who took the test, but 
whose scores were excluded from reporting, and students who did 
not take the test. Of those students who took the test, the test 
scores of 4,048 students were excluded from reporting. Of these, 
378 were bilingual, 3,637 were special education, and 33 were 
both bilingual and special education students. An additional 2,012 
students did not take the test, perhaps because of special educa- 
tion or bilingual status. 



2 3,362 of the 3,629 students who passed Summer Bridge were 
promoted, 131 were retained, 134 left the system, and 2 entered 
Transition Centers. 



3 1 ,200 of the 3,645 students who failed Summer Bridge were pro- 
moted, 2,324 were retained, 1 1 8 left the system, 2 moved into non- 
graded special education, and 1 entered a Transition Center. 



4831 of the 1 ,588 students who did not take the test in Sum- 
mer Bridge were promoted, 580 were retained, 173 left the 
system, 3 moved into non-graded special education, and 1 
entered a Transition Center. 



5 The numbers in this column do not add up to 31 ,385 because 64 
students moved Into non-graded special education between two 
semesters. 



6 The numbers in this column do not add up to exactly 3,581 for the 
following reasons: 1) 141 of the retained 6th graders were enrolled 
in the fall semester, but were not enrolled in the system the follow- 
ing spring semester when the test was re-taken; 2) 8 of the stu- 
dents moved Into non-graded special education between the fall 



and spring semesters; 3) 192 of the students were promoted mid- 
year to 7th grade; and 4) 1 4 students were reclassified into a grade 
other than 6th or 7th between the fall and spring semesters. In 
some cases this was most likely a recording error in the school 
records. This figure traces those 3,226 students that were reclas- 
sified as repeat 6th graders in fall 1998. 



7 The test scores of 426 students were excluded from reporting. 
Of these, 9 were bilingual, 416 were special education, and 1 
was both bilingual and special education. An additional 155 stu- 
dents did not take the test, perhaps because of special or bilin- 
gual status. 



8 451 of the 484 students who passed Summer Bridge 1 998 were 
promoted, 16 were retained, 13 left the system, and 4 entered 
Transition Centers. 



9 207 of the 553 students who failed Summer Bridge were pro- 
moted, 319 were retained, 21 left the system, and 6 moved into 
Transition Centers. 



10 235 of the 404 students who did not take the test at Summer 
Bridge were promoted, 124 were retained, 41 left the system, 1 
was reclassified as non-graded special education, and 3 moved 
into Transition Centers. 



11 The numbers In this column do not add up to exactly 3,581 for 
the reasons listed in Footnote 6, and because 10 students were 
reclassified into non-graded special education between the two 
semesters. 



Retained 6th Graders in Spring 1998 



Spring 1998 Summer 1998 Fall 1998 Fall 1998 

Test results 1^®®^ results Actions Summary 
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Figure 1-4 



1 997-1 998 Test and Retention Results 
All 8th Graders in Spring 1997 



Spring 1997 Summer 1997 Fall 1997 

Test results Test results Actions 



Fall 1997 
Summary 






21,712 

Promoted 
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1,135 

Transition Center 


(yiCBspaa® 



28,812^ 
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1997-1 998 Test and Retention Results 

All 8th Graders in Spring 1997 

Details 
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1 The excluded category includes students who took the test, 
but whose scores were excluded from reporting, and students 
who did not take the test. Of those students who took the test, 
the test scores of 3,868 students were excluded from report- 
ing. Of these, 353 were bilingual, 3,491 were special educa- 
tion, and 24 were both bilingual and special education students. 
An additional 2,112 students did not take the test, perhaps 
because of special education or bilingual status. 

2 2,266 of the 2,488 students who passed the Summer Bridge 
ITBS were promoted, 31 were retained, 185 left the system, 
1 moved into non-graded special education, and 5 entered 
Transition Centers. 



3 638 of the 2,600 students who failed Summer Bridge were 
waived and promoted, 1,046 were retained, 144 left the sys- 
tem, and 772 entered Transition Centers. 

4 627 of the 1 ,291 students who did not take the test in summer 
Bridge were promoted, 228 were retained, 229 left the system, 
1 moved into non-graded special education, and 206 entered 
Transition Centers. 

5 The numbers in this column do not add up to exactly 28,812 
because 187 students moved into non-graded special educa- 
tion between the two semesters. 

6 The test scores of 192 students were excluded from report- 
ing. Of these, 5 were bilingual, 187 were special education, 
and none were both bilingual and special education students. 
An additional 478 students did not take the test, perhaps be- 
cause of special education or bilingual status. 



7 These students were no longer active 8th graders by 
spring 1998. 



8 84 students were promoted to 9th grade in January. 6 of those 
84 students had passed a retest, 62 did not take the test, and 
16 failed the test. While 224 8th graders met the standard in 
January 1998, all but 6 of these students remained in Transi- 
tion Centers or elementary schools until the following fall. 

9 The numbers in this column do not add up to exactly 2,990 
because 26 students were reclassified into a grade other than 
8th or 9th between the fall and spring semesters, perhaps due 
to a recording error in the schools. The numbers in this column 
trace those 2,964 students who were coded as retained 8th 
grade or Transition Center students in fall 1998. 

10 282 of the 296 students who passed Summer Bridge 1998 
were promoted, 5 were retained, and 1 3 left the system, and 1 
entered a Transition Center. 



1 1 38 of the 51 8 students who failed Summer Bridge were pro- 
moted, 7 were retained, 18 left the system, 411 moved into 
Transition Centers. 



12 87 of the 274 students who did not take the test at Summer 
Bridge were promoted, 12 were retained, 97 left the system, 1 
was reclassified as non-graded special education, and 77 
moved into Transition Centers. 



13 The numbers in this column do not add up to exactly 2,990 
for the reasons listed in Footnote 6, and because 23 of the 
students moved into non-graded special education by fall 1998, 
and 96 students were no longer classified as 8th graders or 
Transition Center students in spring 1998. 




Retained 8th Graders in Spring 1998 

Spring 1998 Summer 1998 Fall 1998 

Test results results Actions 



Fall 1998 
Summary 
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CHARTING REFORM IN CHICAGO 



How Many Students Made the Test Cutoff 
by May 1997? 

Main finding: Of the students who were included under 
the policy, half of third graders, 65 percent of sixth grad- 
ers, and 72 percent of eighth graders met the promotional 
criteria in May 1997. 

The first component of CPS s effort to end social pro- 
motion is that students in the third, sixth, and eighth 
grades had to reach a minimum test score in both 
mathematics and reading by May, when testing oc- 
curred, or they were required to participate in Sum- 
mer Bridge. The promotional criteria were based on 
students’ ITBS scores, reported in Grade Equivalents 
(GEs) according to national norms. Since testing oc- 
curred in early May, eight months after the beginning 
of the school year, a student who is testing at national 
norms would receive a score of their grade plus eight 
months. Thus, a third grader is at national norms in 
May if his or her ITBS score is 3.8. The minimum 
test-score cutoff for third graders was set at 2.8, one 
year below grade level. As seen in the first column of 
Figure 1-2, in 1997 only half of included third grad- 
ers met that minimum test score in both reading and 
mathematics by the end of the school year. 

Students in the sixth and eighth grades faced more 
lenient cutoff points. In 1997, sixth graders needed to 
reach a 5.3 in reading and mathematics, a year and a 
half below grade level, to be promoted. Eighth graders 
needed to achieve a test-score minimum of 7.0, which 
is 1 .8 years below grade level, in order to be promoted. 
The cutoff for promotion in eighth grade was increased 
to 7.2 in 1998, and to 7.4 in 1999. 

How Many Students Attended Summer Bridge and 
Passed the Promotional Criteria? 

Main finding: In 1997, more than 80 percent of stu- 
dents who failed the promotional criteria during the school 
year attended Summer Bridge and were retested at the 
end of the summer. More than one-third of third graders 
and approximately 40 percent of sixth and eighth graders 







who failed the promotional criteria in May passed in 
August 1997. Thus, by the end of August, 68 percent of 
third graders, 79 percent of sixth grades and fully 83 per- 
cent of eighth graders had met the minimum cutojfand 
were promoted to the next grade, 

The second component of GPS’s efforts is a manda- 
tory Summer Bridge program for students who do not 
meet the promotional criteria during the regular school 
year. Summer Bridge provides these students with a 
second chance to meet the test cutoff and be promoted 
to the next grade. At the end of the Summer Bridge 
program, students are subject to the same promotional 
criteria as during the school year. In 1997, approxi- 
mately 27,000 students in the third, sixth, and 
eighth grades who were included under the policy 
failed to meet the test cutoff in both subjects. Of 
those, 22,111 were then retested at the end of the 
summer. The second columns in Figures 1-2, 1-3, 
and 1-4 show the results for students who partici- 
pated in Summer Bridge. 

The second chance in Summer Bridge substantially 
raised the proportion of students who ultimately met 
the promotional test cutoff in both subjects. As docu- 
mented in Figure 1-2, 51 percent of included third 
graders met the promotional criteria in May 1997, and 
an additional 17 percent did so over the course of the 
summer, so that 68 percent of included third graders 
met the test cutoff for promotion before entering the 
fourth grade. Among sixth graders, the proportion of 
included students who met the promotional criteria 
increased from 65 percent in May to 79 percent in 
August (Figure 1-3). Among eighth graders, 72 per- 
cent passed in May, while fully 83 percent passed by 
the end of Summer Bridge (Figure 1-4). 

What Happened to Students Who Did Not Meet the 
Test-Score Criteria? 

Main findings: In 1997, about 20 percent of third grad- 
ers, 12 percent of sixth graders and 10 percent of eighth 
graders were retained,^^ The proportion of students who 
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failed to meet the promotional criteria does not match 
the proportion actually retained because nearly one'third 
of students who failed to meet the criteria in 1997 were 
nevertheless promoted to the next grade. 

In August 1 997, CPS faced the decision of whether to 
retain students who did not meet the promotional test 
cutoff or to waive some of these students, promoting 
them despite their test scores. The third columns of 
Figures 1-2, 1-3, and 1-4 show the promotional out- 
comes for students in 1997 in each grade. At the end 
of August, a total of 16,744 students were eligible for 

Ttie yse of waivers may refieci a flex- 
ilrility that is essential when applying a 
policy that has soch important effects on 
students' school careers. 

retention under the policy. Of those, 10,1 19 were re- 
tained, and 5,793 were promoted to the next grade.’^ 
Waivers are an important but contentious policy 
lever in high stakes testing. From one perspective, 
waivers could be viewed as an effort to “get around 
the policy” and weaken its effect. Another perspec- 
tive, however, is that waivers should be expected given 
the diversity of CPS students. The use of waivers may 
reflect a flexibility that is essential when applying a 
policy that has such important effects on students" 
school careers. CPS has taken the position that waiv- 
ers or promotions based on more inclusive criteria are 
useful mechanisms by which to pursue the benefits of 
high stakes testing while considering special circum- 
stances and other indicators of student performance. 

During August 1997 and 1998, waivers were given 
by district superintendents on the basis of appeals by 
principals. In 1997 and 1998, the Guidelines for PrO' 
motion in the Chicago Public Schools did not specify 
criteria for waivers beyond noting that Regional Edu- 







cational Officers (REOs) would consider a student s 
past academic performance in addition to test scores. 

In practice, the specific criteria for waivers were de- 
cided each year by the REOs. Criteria for waivers in- 
cluded special circumstances, such as limited English 
proficiency, health problems, test scores that were very 
close to the cutoff, or additional evidence through 
grades and attendance that the student should be 
promoted. 

In 1997, approximately one-third of third graders 
who failed to meet the promotional criteria in May or 
August were promoted.’^ We can infer that most of 
these promotions were due to waivers granted by 
REOs. Using this method, the waiver rates for sixth 
and eighth graders were even higher. Forty percent 
of sixth and eighth graders who were included un- 
der the policy and did not leave the school system 
were promoted to the next grade despite not meet- 
ing the promotional test score for their grade. 

Promoting one-third of students who did not meet 
the test-score cutoff significantly reduced the propor- 
tion of students in each grade who were retained. 
Among sixth graders, for example, the difference be- 
tween failure and retention rates was substantial. As 
documented in Figure 1-3, more than 20 percent of 
included sixth graders did not meet the test cutoff by 
August 1997, but only 12 percent were retained. 

What Happened in the Second Year for Those Students 
Who Were Retained in Third and Sixth Grades? 

Main finding: Only about one-third of retained third 
and sixth graders in 1997 were able to make the promo- 
tional test cutoff by May 1998. Ultimatelyy 2y365 of 
4y796 retained third graders were required to go to sum- 
mer school a second time. Even after two years in the 
same grade and as many as four chances to pass the testy 
only 43 percent of third graders and 47 percent of sixth 
graders who were retained in 1997 managed to raise their 
scores high enough to meet the test criteria by the end of 
the summer of 1998. Despite low passing ratesy over two- 
thirds of retained students in these grades were promoted 
the nextyeary largely due to waivers. 

2Z 
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Figure 1-5 



Summary Table: Progress of the 1997 Retained 
Students by Fall 1998 





Third 

grade 


Sixth 

grade 


Eighth 

grade 


Retained or Transition 
Center, fall 1 997 


5,551 


3,581 


2,990 


Passed in January 1998^ 






164 (5%) 


Passed in January or 
May 1998 


1 ,764 (32%) 


1 ,204 (34%) 


754 (25%) 


Passed by August 1998 


2,381 (43%) 


1 ,688 (47%) 


1,119(34%) 


Promoted, fall 1998 


3,731 (67%) 


2,491 (69%) 


1,547 (52%) 



^ Students who were in Transition Centers were given a third chance to meet the test score 
cutoff in January 1998. Those who passed remained in Transition Centers for the year. 
They are counted as promoted in fall 1 998. 

Note: There are two categories of students not reported: the percentage of retained 
students who transferred out of the school system during the school year and the percent 
who were excluded from the policy in their retained year. See Figures 1-2, 1-3, and 1-4 for 
more detail. 



Third and sixth graders who were retained at the end of the summer in 

1997 were subject to the same process in 1998. First, some of these stu- 
dents were exempted from the policy the next year by being placed in one 
of the exclusion categories. Second, those students who were not excluded 
were again required to take the ITBS and meet the same test criteria at the 
end of the school year. Third, students who did not pass were required to 
participate in a second year of summer school and were given a fourth 
chance to meet the promotional test cutoff Finally, students who did not 
meet the cutoff in August 1998 could be retained a second time. Figure 1- 
5 summarizes the progress of retained students though this promotional 
process during their second year in the grade. 

Approximately one in ten third and sixth graders who failed to make the 
cutoff and were retained in 1997 were excluded from the policy in the 

1998 testing year.^’ One concern about the policy is that it could provide 
incentives for schools to place students in special education rather than 
have them face a second retention. However, given that retained students 
are among the highest risk students in the Chicago Public Schools, an 
exclusion rate of 10 percent the next year does not suggest widespread use 
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of placing students in special edu- 
cation as a way to avoid the policy. 

Among those 4,796 retained 
third graders who were still in- 
cluded under the policy in 1998, 
2,365 were required to participate 
in a second year of Summer 
Bridge. The majority, 1,786, did, 
and 617 passed. The performance 
of retained sixth graders was only 
slightly better. In the end, less 
than half of those third and sixth 
graders initially retained in 1997 
who were again included under 
the policy were able to raise their 
test scores tp the promotional cut- 
off after four times through the 
ITBS in that grade.^^ 

In August 1998, many re- 
tained students who did not pass 
the test cutoff their second time 
through the policy were pro- 
moted anyway, presumably be- 
cause they received waivers. 
Approximately 38 percent of re- 
tained third graders and almost 
half of retained sixth graders who 
were again subject to and failed 
to meet the promotional criteria 
were promoted in August 1998. 
This meant the majority of re- 
tained students in 1997 were pro- 
moted to the next grade for the 
1997-1998 school year. 

At the end of the summer of 
1 998, CPS had to decide whether 
to double retain those students 
who did not meet the promo- 
tional criteria. This was a contro- 
versial decision. Double retaining 
students almost guarantees that 
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they will have to attend a Transition Center at some 
point because they will turn 15 before reaching eighth 
grade. The practice of double retaining students is so 
rare that we know very little about how double reten- 
tion may impact a student s attitudes and performance 
in school. In the fall of 1998, 1,108 third graders 
and 522 sixth graders were retained a second time 
(see final columns in Figures 1-3 and 1-4). 

What Happened to Eighth Graders Who Were Re- 
tained or Attended Transition Centers? 

Main finding: Passing and promotion rates in the sec- 
ond year were lowest among eighth graders and Transi- 
tion Center students. Only 27 percent of retained or 
Transition Center eighth graders met the promotional cri- 
teria by May 1998. Approximately 38 percent had raised 
their test scores to the test cutojf by August 1998, com- 
pared to 47 percent of sixth graders. This occurred despite 
the fact that Transition Center students had an additional 



chance to pass the test in January 1998. Overall, approxi- 
mately 16 percent of eighth graders retained in the fall of 
1997 had dropped out by fall 1998. Another 375 eighth 
graders had dropped out before the official retention or 
promotion decision in 1997. 

Tracking the progress of eighth graders who were re- 
tained in an elementary school or sent to Transition 
Centers in fall 1997 is complicated because Transition 
Center students were given a third chance to pass the 
promotional test cutoff in January 1998. This was the 
first time CPS experimented with mid-year testing, a 
practice that has now been instituted in all three pro- 
motional gate grades. In January 1998, a total of 1,100 
Transition Center students took the ITBS. One hun- 
dred sixty-four of these students raised their test 
scores in both subjects enough to be promoted (see 
Figure 1-5). 

Part of the reason that second-year passing rates were 
lower among eighth graders who were retained in 1 997 
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is that many either left the system or dropped out. Between fall 1997 and 
fall 1998, 449 retained eighth graders and Transition Center students left 
the system by moving or transferring to another school. This leave rate of 
15 percent is slightly higher than the rate of 12 percent for all CPS eighth 
graders in 1997. Among those who remained in eighth grade for a second 
time or were sent to Transition Centers, 343 students dropped out during 
the school year and another 123 dropped out by fall 1998. This results in 
a 16 percent dropout rate among officially retained students. 

Results for the Second Hhar, 1 997-1 998 

S o far this section has examined the impact of the promotional policy on 
CPS students affected by the first year of Chicago’s efforts to end social 



Figure 1-6 



Summary Table: Passing and Retention Rates 
for 1997 and 1998 

First Time Third, Sixth, and Eighth Graders 





Third 

Grade 


Sixth 

Grade 


Eighth 

Grade 


Passed 


1997 


1998 


1997 


1998 


1997 


1998 


Proportion passed in May 


51% 


61% 


65% 


72% 


72% 


72% 


Summer results: Propor- 
tion of students who 
failed in May but 
passed in August 


35% 


30% 


41% 


41% 


39% 


45% 


Proportion passed in 
May or August ‘ 


68% 


72% 


79% 


83% 


83% 


85% 


Retained 


Proportion retained 


20% 


21% 


12% 


11% 


10% 


10% 


Did Not Pass but Promoted 


Proportion promoted of 
those students who did 
not meet the cutoff 


34% 


21% 


40% 


29% 


40% 


31% 



Note: This table is limited to students who were included in testing. Students who were in 
bilingual education fewer than three years or who were in special education are not repre- 
sented. Thus, the proportion retained does not include students who were retained for 
other reasons and who fell into one of the exclusion criteria. 





promotion in all three promo- 
tional gate grades. What hap- 
pened during the second year in 
which the policy was imple- 
mented? There are several reasons 
why we might expect passing rates 
to increase in 1998. First, incen- 
tive effects for students to work 
harder should be higher in the sec- 
ond year after students have had 
the experience of being in the 
classroom with others who were 
retained. Second, we expect that 
it might take schools and teach- 
ers time to adjust instruction to 
prepare students better for the 
ITBS. And third, as noted in the 
previous section, many more 
schools received extra resources 
during the 1997-1998 school 
year in the form of the Light- 
house after-school program. 

How Did Passing, Waiver, and 
Retention Rates Change from 
1997 to 1998? 

Main finding: Passing rates dur- 
ing the school year improved in both 
the third and sixth grades during 
the second year of the policy. The 
proportion of included third grade 
students who scored a minimum of 
2.8 increased from 51 percent to 61 
percent from May 1997 to May 
1998. Among included sixth grad- 
ers, the passing rate increased from 
65 to 72 percent during the same 
time period. The proportion of 
students who were retained did 
not decline in 1998, however, 
largely because fewer students re- 
ceived waivers. 
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Figure 1-6 compares the passing, retention, and 
waiver rates in 1997 and 1998, the second year in 
which students in all three grades were held to the 
promotional criteria.^"^ Detailed flow charts for 1998 
are included in the Appendix. In 1998, the propor- 
tion of students who met the minimum test-score cut- 
off for promotion by the end of the school year 
increased in the third and sixth grades, but not in eighth 
grade. In 1998, the test-score cutoff for eighth graders 
was raised from 7.0 to 7.2 in both subjects, diminish- 
ing any improvement in passing rates in that grade. 
However, passing rates in Summer Bridge in 1998 were 
much higher among eighth graders. Thus, after Sum- 
mer Bridge, passing rates in 1998 were slightly higher 
in all three grades. 

Despite higher passing rates, the proportion of stu- 
dents retained did not decrease between 1 997 and 1 998 
largely because more students who did not meet the 
standards were retained. For example, in 1997, third 
graders who failed to meet the promotional criteria 
and stayed in the CPS system had a 34 percent chance 
of being promoted, compared to a 21 percent chance 
in 1998. This trend, shown in Figure 1-6, suggests 
that in the first year of the policy. Regional Education 
Officers were more lenient in granting waivers. It will 
be important to track waiver rates over time since this 
trend contradicts the administrations proposal to move 
toward rather than away from using more inclusive 
criteria for promotion at the end of the summer. 

What Have We Learned? 

T his section has highlighted three important aspects 
of the promotional policy which we will look at in 
further detail in the next two sections. First, we found 
that in both 1997 and 1998 the majority of students 
in the third, sixth, and eighth grades were able to meet 
the promotional criteria for their grade by reaching 
the minimum test cutoff in May or after the Summer 
Bridge program. In the first year, passing rates during 
the school year were relatively low, particularly in the 
third grade, where only half the students were able to 
reach a 2.8 in reading and mathematics by May 1997. 



This rate improved by May 1998, suggesting that a 
year of implementation and the addition of Lighthouse 
allowed schools to meet testing goals better. In both 
years, the Summer Bridge program allowed many more 
students to reach the promotional test cutoff so that 
by the end of August, over two-thirds of third graders 
and nearly 80 percent of sixth and eighth graders had 
raised their test scores enough to be promoted. In the 
next section we examine how these passing rates var- 
ied by how far behind students were when they en- 
tered the third, sixth, and eighth grades and compare 
their performance to that of a prior group of students. 

Second, the use of waivers substantially reduced the 
proportion of students who were retained. Even with 
waivers, however, retention rates were high in the third 
grade, where 20 percent of included students were re- 
tained. The decision to retain students is the most con- 
troversial aspect of this promotional initiative. Even if 
the threat of retention produces benefits for those who 
are promoted, the continued feasibility of this initia- 
tive rests on whether CPS teachers and schools find 
ways to address the poor performance of those who 
do not meet the test-score criteria. The lack of progress 
of the first group of retained students under this ini- 
tiative is troubling. After two times through the policy, 
less than half of the students who were retained in 
1997 were able to raise their scores to meet the pro- 
motional cutoff In the next section, we look more 
closely at the performance of retained students in 1997 
by comparing their testing trends to those of students 
in 1995 who were socially promoted. 

Finally, administrative decisions about who should 
be included under the policy substantially shape what 
it means to end social promotion. Many students were 
initially exempted from the policy, and waiver rates, 
particularly in the first year, were relatively high. If we 
consider students who were initially excluded and those 
who were later promoted after failing to meet the cri- 
teria, nearly 40 percent of third graders and 26 per- 
cent of sixth graders in 1997 were not held to the 
test-score cutoff In the last section of this report, we 
examine how these exemption and waiver rates shaped 
racial differences in the effects of the policy. 
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A central premise in CPS s effort to end social promotion is that by 
setting standards and providing extra instructional time to stu- 
dents during the school year and summer, more students will meet 
the minimum test-score cutoffs for their grade. We evaluate this premise 
by comparing the proportion of students who met the cutoff in May 1997 
and May 1998 with results from CPS students in May 1995 who were not 
subject to the promotional criteria. It is hard to evaluate changes in pass- 
ing rates without knowing how many students might have been at risk 
under this policy. If most third, sixth, and eighth graders entered these 
grades with test scores already close to the cutoffs, then getting the major- 
ity of students to pass would not be very difficult. We begin by looking at 
how many CPS third, sixth, and eighth graders were actually at risk of 
retention given the promotional criteria set in 1997. We define risk ac- 
cording to the test-score gains a student would have to make in one year in 
order to meet the minimum test cutoff for promotion. 



How Many Students Were at Hisl( under the Policy? 

Main finding: The initial promotional criteria established by CPS were mod- 
est — a year below grade level for third graders, a year and a half below grade 
level for sixth graders, and a year and eight months below grade level for eighth 
graders. Despite these relatively low test-score cutoffs, however, many Chicago 
students entered these grades with such low test scores that they would have 
needed above average testing gains in that grade to make the promotional cut- 
off' by the end of the year. Almost half of third graders and close to 40 percent of 
sixth and eighth graders could be considered at risk of not passing the test 
criterion given their prior year's reading scores. 
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Figure 2-1 

How Many Third, Sixth, and Eighth Graders Were 
at Risk of Falling below the Test Score Cutoff 
in May 1997? 



Proportion 

of 

Students 
with that 
Learning 
Gap 


Learning Gap Students Would Have to Make 
Up in a Year in Order to Meet the Cutoff 
(Third, Sixth, and Eighth Graders in 1997) 


Greater 
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1.5 GEs 


1 to 
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Reading 


Third grade 


17% 


15% 


16% 


48% 


Sixth grade 


14% 


14% 


10% 


38% 


Eighth grade 


18% 


9% 


10% 


37% 


Mathematics 


Third grade 


1% 


8% 


21% 


30% 


Sixth grade 


7% 


11% 


16% 


34% 


Eighth grade 


12% 


11% 


14% 


37% 



Sidebar 3 

Why Focus on Reading? 

This section focuses on passing rates — the pro- 
portion of students who met the minimum test 
score cutoff — in reading. There are two reasons 
to focus on reading. First, more students were 
at risk in reading than in math, and when stu- 
dents were at risk in reading, they tended to be 
farther behind. Second, it was harder to make 
up deficits in reading than in math. Thus, as 
we noted earlier (Sidebar 2), the combination 
of more students at risk in reading and lower 
test-score gains meant that many more students 
failed to meet the promotional criteria because 
of their reading scores, both during the school 
year and after Summer Bridge. 
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Intuitively, a student s risk of fail- 
ing to meet the promotional cri- 
teria at the end of the year 
depends on how much the stu- 
dent would have to improve his 
or her test scores during the pro- 
motional gate grade. We looked 
at average testing gains in the 
third, sixth, and eighth grade and 
at the test scores of students in 
the year before the promotional 
gate to determine how many stu- 
dents were at risk under this 
policy. In 1997, for example, 17 
percent of third graders had sec- 
ond grade reading ITBS scores 
below a 1.3. These students 
would have to increase their read- 
ing test scores by over 1.5 Grade 
Equivalents (GEs) in one year — 
over twice the normal test-score 
improvement rate of third grad- 
ers in CPS — to make the promo- 
tional cutoff. We call these 
students high risk. We considered 
students more moderately at risk 
if they would need average to 
above average (.5 to 1.5 GEs) in- 
creases in one year to make the 
promotional cutoff.^^ Using these 
cutoffs, we found that many 
third, sixth, and eighth graders 
were at high or moderate risk of 
not meeting the promotional cri- 
teria in 1997 (see Figure 2-1). 
How many of these students 
managed to close their test score 
gap? We begin by looking closely 
at the performance of sixth grad- 
ers in 1995, 1997, and 1998. We 
then examine results for third and 
eighth graders. 
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Figure 2-2 

Moire Students Are Meeting Test Score Cutoff in 1997 Than in 1995 
Sixth Grade Reading 



High risk Moderate risk 



Low risk 




How far below or above test score cutoff (5.3) at end of fifth grade 



Were More Sixth Graders Meeting the Cutoff in 1997 
Than before the Policy? 

Main finding: In 1997, a higher proportion of sixth grad- 
ers reached a 53 on the ITBS in reading by May than 
did sixth graders in 1995. This holds true across all risk 
categoriesy with the students at highest risk showing the 
largest gains in the proportion meeting the minimum test 
scores for promotion. 



Even if students had very low test scores in fifth grade, 
we might expect some to reach a 5.3 by the end of 
sixth grade either because they had a particularly good 
sixth grade year or because their fifth grade test scores 
were abnormally low by chance. We can adjust for these 
chance effects by comparing the performance of stu- 
dents who faced the promotional criteria in 1997 with 
the performance of sixth graders in 1995. Since sixth 
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graders in 1995 were not subject to the promotional 
test cutoff, their outcomes provide a comparison for 
what we could expect in the absence of the policy. 

The black line in Figure 2-2 shows the proportion 
of sixth graders in May 1995 whose reading ITBS test 
scores were 5.3 or higher by the end of sixth grade by 
the testing gap students faced in that year. For example, 
20 percent of students who entered sixth grade in 1 994 
with the lowest test scores obtained a 5.3 or higher on 
the reading ITBS in May 1995 at the end of sixth 
grade. The green line in Figure 2-2 shows the propor- 
tion of sixth graders in each risk category who made a 
5.3 or higher in reading by the end of the 1996-1997 
school year. Among those with the lowest test scores 
in May 1997 (more than 1.5 GEs below grade level), 
31 percent scored a 5.3 or higher in reading. Thus, for 
this highest risk category, the proportion of sixth grad- 
ers who passed the cutoff in May was 1 1 percent higher 
than in 1995. This trend was true across all risk cat- 
egories. The difference between the 1995 passing rate 
and the 1997 passing rate could be interpreted as the 
additional increase in passing associated with the pro- 
motional policy during the school year. 

It is clear from these numbers that more sixth grade 
students were meeting the promotional cutoff of 5.3 
during the 1996-97 school year than did in 1994-95. 
In the next section we will look at results for third and 
eighth graders. Sorting out the possible explanations 
for this increase is hardly straightforward. We see five 
possibilities. First, there may be a testing or instru- 
mentation effect of the policy, meaning that students 
may simply be taking the test more seriously or may 
be improving their test-taking skills. Second, the test- 
score cutoff may have had a motivational effect on 
students, leading them to study harder and learn more 
during the school year. Third, because of the promo- 
tional criteria, teachers may have changed their instruc- 
tion to focus more on improving students' basic skills 
or may have spent more time working with these stu- 
dents to raise their performance, what we would call 



an instructional effect. Fourth, there may be a positive 
effect from students participating in the Lighthouse 
after-school program, which we would call a program- 
matic effect. Finally, passing rates may have simply 
increased because test scores have been generally ris- 
ing in the Chicago Public Schools for several years. 
We call this a general reform effect.^^ While the previ- 
ous explanations may all be a result of the policy, this 
last effect could be an artifact of underlying trends. At 
this point, we do not have sufficient evidence to evalu- 
ate these competing hypotheses. Subsequent reports 
in this series will seek to untangle more clearly the 
nature of these increases in the proportion of students 
meeting the promotional criteria. 




Who Met the Promotional Cutoffs? 

Main finding: The second chance afforded by the Sum- 
mer Bridge program substantially raised the propor- 
tion of sixth graders who reached the 53 cutoff before 
promotion. 

An important difference between students in May 
1995 and May 1997 was that students in 1997 who 
failed to reach a 5.3 or higher on the ITBS were re- 
quired to go to Summer Bridge and retake the test in 
August. The red line in Figure 2-2 shows the propor- 
tion of sixth graders who managed to meet the cutoff 
by either May or August 1997, after Summer Bridge. 
For example, 31 percent of the sixth graders at highest 
risk in 1997 reached the test cutoff by May. An addi- 
tional 21 percent attended Summer Bridge and scored 
a minimum of 5.3 in reading in August. As a result, 
52 percent of sixth graders in 1997 with the lowest 
reading skills scored 5.3 or higher in reading by Au- 
gust, compared to only 20 percent in 1995. These 
improvements in passing rates after Summer Bridge 
were equally impressive for students in more mod- 
erate risk categories. 
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Kow Many More Students Met 
the Cutoff? A More Rigorous 
Approach to Estimating 

Main finding: Across all sixth 
graders in the 1996-97 school year, 
the proportion who reached a mini- 
mum ITBS reading score of 5.3 
before promotion to seventh grade 
was 20 percent higher in 1997 than 
in 1995. Even after using a statis- 
tical model to correct for abnor- 
mally low test scores at the end of 
fifth grade, the most at risk students 
showed the most improved perfor- 
mance under the policy. The com- 
bination of slightly higher passing 
rates during the school year and a 
big increase in the proportion who 
passed over the summer doubled the 
number of moderate to high risk 
students who reached the minimum 
cutoff between 1997 and 1995. 

The increases in passing rates are 
impressive and suggest that the 
policy may be having a positive 
effect on raising students' scores 
to minimum test cutoffs before 
promotion. In order to look at 
this more systematically, we de- 
veloped a statistical model to es- 
timate each student s risk of not 
meeting the cutoff based on his 
or her growth trajectory over all 
of the years he or she has been in 
the school system, rather than just 
simply using the prior (fifth 
grade) test score. Thus, if a 
student s fifth grade test score was 



Figure 2-3 



Whal Percent of Sixth Grade Students Are 
tVieeting the 5.3 Cutoff? 

Comparing Reading Scores from 1995 and 1997 




May May August May May August May May August 

1995 1997 1997 1995 1997 1997 1995 1997 1997 

High Risk Moderate Risk Aii Students 



abnormally low, the statistical model would correct for that, since it pre- 
dicts a students fifth grade test score on the basis of what his or her test- 
score growth looks like from the second to the fifth grade. We call this the 
predicted fifth grade test score and use that test score to derive a new, more 
reliable, estimate of how much students fall below the sixth grade cutoff 
Again, we are looking only at sixth grade; in the next section we turn to 
results for third and eighth graders. When we use this predicted measure. 
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Figure 2-4 

Wha4 Percert of Third Grade Sfudents Are 
Meeting the 2.8 Cutoff? 

Comparing Reading Scores from 1995 and 1997 




1995 1997 1997 1995 1997 1997 1995 1997 1997 

High Risk Moderate Risk Aii Students 

Note for Figures 2-3, 2-4, and 2-5: The risk categories are created from predicted scores 
based on students' ITBS growth trajectories in all years prior to third grade. High risk 
students are those whose predicted test scores mean they would need to make up over 
1 .5 GEs in a year to make the cutoff. Moderate risk students are those who would have to 
make up .5 to 1 .5 GEs in a year to make the cutoff. 



we find results similar to the descriptive results. Figures 2-3, 2-4, and 2-5 
detail the passing rates of third, sixth, and eighth graders using their pre- 
dicted test scores to calculate their risk. 

Even after statistically adjusting for abnormally low test scores in fifth 
grade, the proportion of sixth graders who scored at least a 5-3 on their 
ITBS in reading by the time they were promoted to seventh grade increased 
from 4 percent to 34 percent among the students at highest risk between 
May 1995 and August 1997, and from 41 percent to 76 percent among 
students at moderate risk. 
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How Do Passing Rates for Third, 
Sixth, and Eighth Graders in 
1997 Compare to 1995 Rates? 

Main finding: There was little in- 
crease in the proportion of third 
graders who met the minimum 
ITBS cutojf in reading of 2.8 be- 
tween May 1995 and May 1997. 
High passing rates in Summer 
Bridge, however, substantially 
raised the proportion of third grad- 
ers who reached 2. 8 on the ITBS 
in reading before the following fall. 
By August 1997 y 71 percent of all 
third graders scored 2. 8 or higher 
on the reading ITBS compared to 
only 58 percent in May 1997 and 
55 percent in 1995. As with sixth 
graders, increases in passing rates 
among eighth graders reflected a 
combination of more students meet- 
ing the cutojf during the school year 
and during Summer Bridge. Only 
66 percent of eighth graders in 
1995 scored a 7.0 or higher in 
reading by May. In 1997 y 78 per- 
cent of eighth graders reached a 
7. 0 by May, and an additional 9 
percent did so over the course of 
the summer. 

Figures 2-3 through 2-5 present 
the testing results for all third, 
sixth, and eighth graders by their 
predicted risk. In both the third 
and eighth grades, the proportion 
of students who reached the mini- 
mum criteria set by CPS before 
the following fall was substan- 
tially higher in 1997 than in 
1995. But while in sixth and 
eighth grades this increase reflects 
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both higher passing rates during 
the school year and students pass- 
ing over the summer, in the third 
grade there appears to be little 
school-year effect. This was true 
across all risk categories. Thus, 
among third graders, almost all 
of the gains in the proportion of 
students meeting the minimum 
test cutoff in August 1 997 can be 
attributed to the effect of Sum- 
mer Bridge in raising students’ 
test scores, particularly among 
high to moderate risk students. 

In contrast, increases in the 
proportion of students meeting 
the cutoff by May were greatest 
among eighth graders. Among all 
eighth graders, the proportion 
who scored 7.0 or more on the 
ITBS in reading was 12 percent 
higher in May 1997 than in 1995. 
Among the high risk eighth grad- 
ers, the proportion who reached 
this cutoff during the school year 
doubled between May 1995 and 
May 1997, from 12 to 26 per- 
cent. These percentages increased 
even more after Summer Bridge. 
Indeed, among the eighth grad- 
ers at high risk, nearly half had a 
reading ITBS score of 7.0 or 
higher before the following fall! 

Why was the increase in school 
year passing rates so much higher 
among eighth graders than 
among third graders? The 1996- 
1997 school year was the first year 
in which third graders faced the 
promotional criteria. It was the 



Figure 2-5 

Whatt Peircenit of Eiglhifh Grade Sfudeofs Are 
Meefiiing the 7.0 Cutoff? 

Comparing Reading Scores from 1995 and 1997 




May May August May May August May May August 

1995 1997 1997 1995 1997 1997 1995 1997 1997 

High Risk Moderate Risk Aii Students 



second year for eighth graders since the CPS began the promotional initia- 
tive in the 1995-96 school year with the eighth grade. Thus, some of this 
difference may be the effect of a second year of implementation. But these 
differences are large and are reflected across all three grades. Why we ob- 
serve greater effects of the policy in eighth and sixth grades is an issue we 
will return to in the interpretative summary of this report. 
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Figure 2-6 

Sixth-Grade Passing Rates by Prior Year Risk Category 
May 1995, 1997, and 1998 
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To What Extent Were 1998 Passing Bates for Third 
and Sixth Graders Better? 

Main finding: The proportion of sixth graders who met 
the promotional criteria by the end of sixth grade in 1998 
increased across all risk categories. For third graders^ im^ 
provements in passing rates were concentrated among more 
moderately at risk students. The proportion of high risk 
third graders who were able to raise their reading test 







scores to 2.8 by the end of third grade changed little be^ 
tween May 1997 and May 1998. 

In the previous section, we documented that in the 
second year of the policy, the proportion of students 
who met the minimum ITBS cutoffs for promo- 
tion improved in both the third and sixth grades. 
To what extent was this improved performance 
shared across students? 
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Figure 2-6 compares the proportion of sixth grad- 
ers in May of 1995, 1997, and 1998 who met the 
promotional cutoff of 5.3 by the end of the school 
year by level of risk on entry into sixth grade. As shown, 
the 1998 (orange) line is above both the 1997 (green) 
and 1995 (black) lines. This means that passing rates 
among sixth graders were higher in almost every risk 
category in 1998 than in 1997 and in 1995. Increases 
in passing rates were observed in almost every risk cat- 
egory. Thus, not only do we find evidence of a school- 
year effect of the policy in increasing the proportion 
of students who met the cutoffs by May, we also find 
improvements in passing rates in the second year of 
the policy. For example, between 1995 and 1998, the 
proportion of sixth grade students who increased their 
reading ITBS scores by 1.5 GEs or more to meet the 
cutoff was 37 percent in 1998 compared to only 20 
percent in 1995. 

Results for third graders are more mixed. Across all 
third graders, the proportion who met or exceeded the 
2.8 reading cutoff by May rose 10 percent from 1997 
to 1998 (see Figure 1-6 on page 20). But this improve- 
ment occurred because students at moderate risk were 
doing better. As seen in Figure 2-7, there was very little 
improvement over pre-policy (1995) trends in the pro- 
portion of high-risk third graders who were able to 
meet the cutoff Indeed, among the highest risk stu- 
dents, the 1998 (orange) passing rate is actually below 
the 1997 (green) and 1995 (black) rates. Greater im- 
provement among more moderate-risk students could 
mean that teachers are beginning to triage their efforts 
to raise third graders’ skills to the minimum cutoff It 
may also mean that instruction and programs in the 
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third grade are improving the chances of passing for 
moderate-risk students, but are leaving those with the 
poorest skills behind. 

A Comparison of Two-Ysar ITBS 
Achievement Trends for 1 995 
and 1997 Students 

U ltimately, whether the CPS policy is considered a 
success depends upon students’ long-term perfor- 
mance. The 1996-1997 group is the first group of stu- 
dents who faced promotional criteria for which we can 
examine testing gains both in the year prior to and the 
year following promotion or retention. In this section, 
we compare 1997 students’ test-score trends over two 
years to those of students in 1995 for third and sixth 
graders. Eighth graders who are promoted to high 
school take a different test, the Test of Achievement 
Proficiency (TAP). Because TAP and ITBS scores are 
not strictly comparable, we cannot simply compare 
test-score gains in the year after promotion or reten- 
tion. We emphasize that the statistics presented in this 
section are descriptive. Forthcoming technical reports 
will offer more specific estimates of the effect of the 
policy on test-score gains for students in the pre- 
testing year, in Summer Bridge, and after retention 
or promotion. We look at the passing rates for the 
sixth graders first, then follow with those for third 
and eighth graders. 

What Were the STBS Achievement Trends among 
Students Who Met the Test Criteria in May? 

Main finding: Students who reached the sixth grade test' 
score cutojf by the end of the school year in 1997 had 
ITBS achievement gains between fifth and seventh grade 
that were comparable to students in the 1995 group who 
would have met the cutojf The dijference between 1995 
and 1997 was that a higher proportion of sixth graders 
were now represented in this group. 
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Figure 2-7 

ThiB-d-Girade Passing Rates by Prior Year Risk Category 
May 1995, 1997, and 1998 
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How far below test score cutoff (2.8) at end of second grade 



In May 1995, 63 percent of sixth graders had ITBS 
scores that would have met the minimum cutoff of 
5.3. The dashed black line in Figure 2^8 documents 
these students' test-score trends from May in fifth grade 
to May in seventh grade. This is our comparison 
group — students who would have met the test-score 
cutoff in the sixth grade had the policy been in place. 
The green line shows the test-score trends for students 
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in the 1997 group who met the minimum cutoff in 
May. Both of these groups looked similar at the end of 
fifth grade. Both groups increased their ITBS scores 
by approximately two Grade Equivalents (GEs) over 
the two years between May of fifth and May of sev- 
enth grade. But as we saw in the previous section, 70 
percent of sixth graders passed the cutoff at the end of 
the school year in 1997. Thus, a higher proportion of 
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students are represented in the May passing line in 
1997 than in 1995, and these students had similar 
growth trajectories over the two years. 

What Were the ITBS Achievement Trends for Sixth 
Graders in 1995 Who Were Socially Promoted? 

Main finding: Sixth graders who were socially promoted 
in 1995 were falling behind their counterparts. 

The dashed grey line in Figure 2-8 shows the two-year 
test-score trend for our second comparison group, the 
37 percent of sixth grade students in May 1995 who 
did not meet the minimum test score of 5.3 used in 
May 1997 as the promotional cutoff. These are the 
""social promotes ^ — students who were promoted de- 
spite performance that was substantially below grade 
level. The average end-of-fifth-grade test score of the 
“social promotes” — a 3.9 — ^was almost two years be- 
low grade level. This group had very small test-score 
gains in the sixth grade, and while their reading ITBS 
scores increased 1 .2 GEs on average in seventh grade, 
they did not make up for their lower than average start- 
ing point in fifth grade. Between the end of fifth grade 
and the end of seventh grade, students who did not 
make the minimum 5.3 test score in May 1995 had 
two-year test-score gains of 1.5 years on average, com- 
pared to test-score gains of approximately 2.0 years on 
average for their counterparts who met the cutoff. 
Thus, the 1995 group of socially promoted students 
were 1 .9 years on average below grade level at the end 
of fifth grade, but were fully 2.4 years below grade 
level by the end of seventh grade. This testing trajec- 
tory demonstrates why social promotion became such 
a concern in Chicago. Clearly, there was a large group 
of students who were falling even further behind as 
they moved through elementary school. 

Did Summer Bridge Pay Off? 

Main finding: Sixth graders who failed to meet the read- 
ing cutoff at the end of sixth grade but passed after Sum- 



mer Bridge had comparable two-year test-score gainSy on 
average, to their counterparts who made the test cutoff in 
May. Unfortunately, these students did not make up suf- 
ficient ground. As a result, two years later they were at 
risk of not meeting the eighth grade cutoff. 

As we saw previously, in 1997 an additional 13 per- 
cent of sixth graders achieved the promotional cutoff 
score by August, after Summer Bridge. The blue line 
in Figure 2-8 shows the average growth trend over two 
years for these students. Results for this group are 
mixed. First, students who failed to meet the reading 
ITBS cutoff in May and passed after Summer Bridge 
had very poor test-score gains in sixth grade and dra- 
matic increases in Summer Bridge. This suggests that 
their May test scores might have been abnormally low. 
Moreover, large Summer Bridge increases did not trans- 
late into substantially better test-score gains in the sev- 
enth grade. Rather, after a bad sixth grade year. Summer 
Bridge promotes had, on average, another weak sev- 
enth grade year. Students who passed after Summer 
Bridge gained only 5 months during seventh grade. 

Even if we consider that these Summer Bridge gains 
may be somewhat inflated because of abnormally low 
test scores in May, it is clear for this middle group that 
test-score gains in Summer Bridge worked to keep 
them on track, but did not allow them to make up 
ground. This means that the sixth graders who passed 
after Summer Bridge entered eighth grade again at risk 
of not meeting the promotional criteria. That is, a sev- 
enth grader with reading test scores of 6.4 would again 
need to make up a year in order to be promoted to the 
next grade. This is well above the testing rate this group 
demonstrated in the prior two school years. It will be 
important to follow these students through their sec- 
ond promotional gate. How much can they draw on 
the prior experience of facing and meeting a test cut- 
off when they must do it again in eighth grade? And, 
how many of these students will be required to par- 
ticipate in a second round of Summer Bridge in order 
to continue to high school? 
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Figure 2-8 

Two-Year Growth in IITBS Reading Scores 
Sixth Graders in 1997 Compared with 1995 




Spring Spring Bridge Spring 

prior year 1995 or 1997 following year 



Students Who Met the 1997 Cutoff Students Who Did Not Meet the 1997 Cutoff 



1995: Attained a score of 5.3 or higher (n = 13,683) 
May 1997: Made promotion cutoff (5.3) (n = 15,876) 
May 1997 Bridge: Made promotion cutoff in August 
(n = 2,626) 



1995: Did not attain a score of 5.3 (n = 9,014) 
Bridge 1997: Did not pass but promoted after 
Summer Bridge (n = 774)^ 

Bridge 1997: Retained after Summer Bridge 
(n = 1,782)^ 



Only students with Summer Bridge scores; 2,91 1 students were 



eliminated from these groups because they did not participate in 
Summer Bridge. 
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How Did Detained and Waived Students Progress? 

Main finding: Both waived and retained sixth graders 
in 1997 did as poorly as those who were socially pro- 
moted in 1995. 

The dark red and yellow lines in Figure 2-8 show test- 
score gains over two years for sixth grade students in 
1997 who did not meet the test cutoff in August after 
attending Summer Bridge.^^ Students who did not 
meet the test criteria in August 1997 look, on aver- 
age, similar to the 1995 social promotes. And, like 
previously socially promoted students, this group had 
very weak sixth grade years. Importantly, retained stu- 
dents in 1997 also made little progress over the sum- 
mer. Because of their relatively poor sixth grade years 
and the fact that they did not make that up in Sum- 
mer Bridge, retained and waived students were falling 
farther behind their counterparts who made the cut- 
off and were promoted. Students who were retained 
in 1997 had two-year test-score gains of 1 .4 years com- 
pared to gains of 1 .6 for students who were waived, 
and gains of 1.5 for students in 1995 who were so- 
cially promoted. The difference is that retained stu- 
dents faced the test cutoff again in May 1998. In the 
first section, we noted that passing rates for retained 
students in the second year were very low. Figure 2-8 
demonstrates why. At the end of the second time 
through sixth grade, the average retained sixth grade 
student had a reading ITBS score of only 5.2, still 
below the promotional test criterion. 

What Were the Results for Third Graders? 

Main finding: Third graders who made the test cutoff in 
May 1997 had test-score gains in the year after promo- 
tion similar to those of students who would have made 
the cutoff in May 1995. Those who passed after Summer 
Bridge had substantial test-score gains during Summer 
BridgCy but again performed more poorly during the fol- 
lowing school year. The performance of third graders who 



were retained in 1997 was worse than that of third grad- 
ers in 1995 who had similar ITBS scores, but were pro- 
moted. Third graders who were retained in 1997 had 
very poor third grade years and only slightly better test- 
score gains their second time through third grade. 

In general, the pattern of results among third graders 
is quite similar to that discussed above for sixth grad- 
ers. As seen in Figure 2-9, the average third grader in 
both May 1995 and May 1997 who made the promo- 
tional cutoff of 2.8 had test-score gains of 2.0 over 
two years. Similarly, third graders in 1997 who par- 
ticipated in Summer Bridge and passed at the end of 
the summer gained 1.9 GEs over the two years. 

Third grade results differ from those observed in 
the sixth grade regarding trends in the test-score gains 
of retained students. Unlike retained sixth graders in 
1997, who had gains in their second year comparable 
to their promoted counterparts from 1995, retained 
third graders had another relatively poor year the sec- 
ond time through third grade. Over two years, third 
grade students who were retained in 1997 increased 
their ITBS reading scores by only 1.2 GEs, compared 
to approximately 1.6 years on average for students in 
1 997 who were waived, and 1.5 for previously socially 
promoted students (see Figure 2-9). After two years in 
third grade, the average score of retained third graders 
was again below the cutoff of 2.8, explaining why only 
one-third of retained third graders were able to pass 
the promotional criterion in May 1998. This trend 
underscores that third graders who were retained in 
1997 appear to present substantial learning problems. 

What Were the Drepout Rates among Eighth Graders 
in 1995 and 1997? 

Main finding: The one-year dropout rate among eighth 
graders with low skills was slightly higher in 1997 than 
in 1995, although there was no increase in the overall 
dropout rate for the 1997 group. 
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Figure 2-9 

Two-Year Growth Sn JIBS Reading Scores 
Third Graders in 1997 Compared with 1995 




Spring Spring Bridge Spring 

prior year 1 995 or 1 997 following year 



Students Who Met the 1997 Cutoff Students Who Did Not Meet the 1997 Cutoff 

1 995: Attained a score of 2.8 or higher (n = 1 0,689) 1 995: Did not attain a score of 2.8 (n = 9,495) 

May 1997: Made promotion cutoff (2.8) (n = 1 0,971 ) 1 997 Bridge: Did not pass but promoted after 

1997 Bridge: Made promotion cutoff in August Summer Bridge (n = 1,325)^ 

(n = 2,402) 1997 Bridge: Retained after Summer Bridge 

(n = 2,763)’ 

1 

Only students with Summer Bridge scores; 1 ,044 students were 
eliminated from these groups because they did not participate in 
Summer Bridge. 
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A final and important topic in our 
examination of the post-retention 
or promotion year is the question 
of whether more eighth grade 
students are dropping out. We 
documented in Section 1 (see 
Figure 1-4 on page 14) that 16 
percent of eighth graders who 
were retained or sent to Transi- 
tion Centers in 1997 dropped out 
by the fall of 1998. Assessing the 
impact of the social promotion 
policy in this area is tricky because 
prior studies find that students 
with low skills often face high rates 
of failure in high school. In order 
to begin addressing this question, 
we compared the dropout rate 
among students who did not 
meet the minimum test score for 
promotion in 1997 with that of 
students who would not have met 
the test criteria but were socially 
promoted in 1995. As seen in Fig- 
ure 2-10, 8 percent of students 
in 1995 who had ITBS reading 
and mathematics scores below 7.0 
dropped out by the fall of the next 
year. Since these students with 
low skills would have failed to 
meet the test cutoff had it been 
in place, they represent a good 
comparison group for the drop- 
out rate we might expect among 
students who faced the policy in 
1997. The comparable dropout 
rate for eighth graders who failed 
to meet the test cutoff in 1997 
was just slightly higher than 
among the 1995 group. Because 
we are dealing with dropout rates 



Figure 2-10 



Percent off 1995 and 1997 Eighth Graders Who 
Dropped Out by Fall affter the Retention Year 




"ITBS <7.0" is the dropout rate for eighth graders who failed the ITBS cutoff in May 1997 
and those who would have failed to meet the cutoff in 1995 if the policy had been in 
place that year. 



among a small group, there was no appreciable increase in the overall one- 
year dropout rate.^^ This trend in dropout rates among students with the 
lowest skills is troubling, however, and is consistent with the findings of 
prior research that students who are retained are more likely to drop out. 

Subsequent reports will track trends in dropout rates through high school. 
Since the summer between tenth and eleventh grade is the most common 
time for dropping out, results from the 1998-1999 school year will allow 
us to derive a more accurate and complete assessment of whether drop- 
out rates are rising under the promotional policy. 



What Have We Learned? 

A t first glance, the test-score cutoffs set by the Chicago Public Schools 
for promotion might seem low. Nonetheless, a high proportion of CPS 
third, sixth, and eighth graders had such low test scores that they were at 
risk of not meeting those cutoffs. Nearly one-third of third graders and 
approximately one-quarter of sixth and eighth graders had to increase their 
reading ITBS scores by over a year to meet the test criteria in 1997. 

In this light, the performance of students in these grades seems like a 
major success. The proportion of students who raised their test scores to 
the promotional cutoff was substantially higher during the 1996-97 school 
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year than during 1994-95. In the sixth and eighth 
grades, increases reflected both improvement in stu- 
dent performance during the school year and posi- 
tive effects associated with Summer Bridge. In the 
third grade, most improvements in the proportion of 
students who were able to attain a minimum of 2.8 in 
reading occurred during the Summer Bridge program. 

Increases in passing rates were highest among the 
students at greatest risk in grades six and eight but not 
in grade three. Critics worry that this policy sets stu- 
dents with the lowest skills up for failure, placing the 
costs of the policy on the backs of the most vulner- 
able. The statistics presented here substantially com- 
plicate the story. Students with very low test scores 
were retained at higher rates. But among those same 
high-risk students, the proportion who were able to 
meet the minimum cutoff for their grade increased 
from 4 to 34 percent among sixth graders and from 



12 to 49 percent among eighth graders, as shown in 
Figures 2-4 and 2-5 on pages 28 and 29. 

The basic theory of action of the CPS promotional 
initiative is that getting students’ test scores up to a 
minimum cutoff will lay the basis for long-term school 
success. In this respect, the evidence from the first 
group of students who experienced this policy is mixed 
and inconclusive. One way of viewing the results pre- 
sented in this section is that the promotional policy 
has taken the very large group of students who used to 
be socially promoted and sorted those students into 
three categories. First, more students are meeting the 
test criteria during the school year. These students now 
share testing trajectories comparable to their counter- 
parts who would have met the cutoffs in May 1995. 

Second, more students are meeting the minimum 
cutoffs after participating in Summer Bridge. In the 
sixth grade, for example, 13 percent of 1997 students 
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failed to make the promotional criteria in May but 
did so after Summer Bridge (see Figure 2-4 on page 
28). The story for these students, however, is more 
mixed. These students were placed into Summer Bridge 
because they had relatively poor test-score gains in sixth 
grade. Summer Bridge worked to jump these students 
forward and, because of large test-score gains in Sum- 
mer Bridge, their two-year ITBS achievement trajec- 
tories were on track. But, the following year they 
returned to ITBS achievement rates similar to their 
prior years. As a result, they actually made up little 
ground over the course of two years. Consequently, 
sixth graders who passed after Summer Bridge were 
again at risk of not meeting the promotional criteria 
in the eighth grade. It appears that the positive effects 
of the Summer Bridge program are not compensating 
for weak instruction or motivation problems during 
the school year. These more marginal students may 
need extra help all along the way, an explanation that 
challenges the assumption that one-shot interventions 
are all that students need. After 1999, we will know 
much more about test-score trends among this smaller 
group having observing their experience through a sec- 
ond promotional gate. 

The third group of students are those who do not 
make the promotional gate and are retained or are 
waived. In the sixth grade in 1997, for example, 17 
percent of students did not meet the promotional test 
cutoff at the end of the school year or after Summer 
Bridge (see Figure 2-4 on page 28), and as we saw in 
the previous section, 12 percent were retained (see Fig- 
ure 1-5 on page 18). Clearly, this is better than the 37 
percent of students who did not meet the cutoff in 
1995 and were socially promoted (see Figure 2-4 on 
page 28). But ITBS achievement trends among this 



group are troubling. Students who were retained do 
not appear through these simple descriptive compari- 
sons to be doing any better than students who were 
previously socially promoted. They may actually be 
doing slightly worse — particularly the third graders. 
The difference is that retained students now have the 



Ttie goal of CPS efforts diireiig ie re- 
toiiiei] year is to address poor perfor- 
mance among stodents wlio do not meet 
ie minimum test cotoff. Iliis goal is 
cleariy not being met 

negative experience of retention, are now over- age for 
grade and are now faced with meeting the promotional 
gate a second time. Clearly, we need to be careful in 
over-interpreting these simple descriptive comparisons 
because we are comparing the performance of all stu- 
dents who were previously socially promoted to the 
much smaller group of students who were retained 
under the policy in 1997. Thus, for third graders, we 
are comparing trends in the performance of the 45 
percent of third graders in 1995 who were socially pro- 
moted to trends among the 20 percent of students in 
1997 who were retained. How comparable these group 
are remains uncertain. Nonetheless, the bottom line is 
clear. The goal of CPS efforts during the retained year 
is to address poor performance among students who 
do not meet the minimum test cutoff This goal is 
clearly not being met. 
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I ike many large urban school systems, the CPS primarily serves minor- 
ity students. In the 1996-1997 school year, 54 percent of the CPS 
student body was African-American, 32 percent was Latino, and 10 
percent was white non-Latino. Many critics of this policy worry that if 
large urban school systems adopt promotional testing policies, minority 
students will be disproportionately affected by retention. This is a particu- 
lar concern for Latino students, who are the fastest rising population in 
the Chicago Public Schools.^"^ Bilingual students — especially recent immi- 
grants — may be particularly at risk for retention because they are trying to 
meet the testing criteria while gaining proficiency in a new language. 

Section 1 of this report examined two sets of administrative decisions 
that affect outcomes under the CPS promotional policy. We looked at the 
criteria determining whether students’ promotional decisions are subject 
to the test-score cutoffs — whether they are included or excluded — and the 
decision to waive students who do not meet the cutoff In this section, we 
examine how these administrative decisions and the performance of stu- 
dents under the policy varied between Latinos and African-Americans and 
between boys and girls during the first year of implementation, 1 996- 1 997. 
We limit our analysis to African-Americans and Latinos because the num- 
bers of non-Latino white and Asian students who are highly at risk under 
this policy are too small for valid comparisons of performance. 
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Figure 3-1 

African-American and Latino Students Excluded 
Third, Sixth, and Eighth Graders, 1997 



African-American Students 




I I Percent not tested [Zl Percent excluded after testing 



Latino Students 




I I Percent not tested LZ! Percent excluded after testing 



Note: 5% and below not labeled. 



How Did Ethnic Differences 
Impact Third Graders 
under the Policy? 

Main finding: In the third grade, 
62 percent of Latino students were 
excluded from the promotional 
policy, compared to approximately 
11 percent of African-American 
students. Among those students in- 
cluded under the policy, African- 
American third graders were more 
at risk of non-promotion and, once 
tested, had poorer passing rates than 
Latinos. The combination of higher 
exclusion rates and slightly higher 
passing rates among Latinos meant 
that African-American third grad- 
ers were 1.67 times more likely to 
be retained in the third grade than 
their Latino counterparts. 

Section 1 of this report docu- 
ments that almost one-third of 
third graders were excluded from 
the policy, largely because they 
were in bilingual education. 
Clearly, these exclusions were 
concentrated among Latinos. 
During the 1996-1997 school 
year, 46 percent of Latino third 
graders were not tested because 
of their participation in bilingual 
education, and among those 
tested an additional 16 percent 
had their test scores excluded 
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Figure 3-2 

African-American and Latino Students at High and Moderate 
Risk of Failing 

Third, Sixth, and Eighth Graders, 1997 



High risk Moderate risk 




□ African-American 



□ Latino 



Note: High risk students are those whose predicted test scores indicate they would need to make up over 1 .5 GEs in a 
year to make the test score cutoff. Moderate risk students are those who would have to make up .5 to 1 .5 GEs in a year 
to make the cutoff. 



from the policy, meaning that their promotional deci- 
sions were not strictly subject to the policy’s set pro- 
motional criteria. As shown in Figure 3-1, a total of 
62 percent of Latino third graders were excluded from 
the policy. 

Among included students, African-American third 
graders were more likely to be at risk of failing to meet 
the promotional criteria. Using a statistical model for 
predicting the test-score gap that students would have 
to make up to meet the promotional test cutoff, 49 
percent of African-American and 44 percent of Latino 



third graders would be considered at moderate to high 
risk under the policy. Another 9 percent of African- 
American third graders and 6 percent of Latino third 
graders were at high risk (see Figure 3-2). In addition, 
African-American students typically had somewhat 
lower testing gains during third grade. For example, 
among students who were moderately at risk of not 
meeting the cutoff, 75 percent of African-American 
third graders versus 63 percent of Latino students failed 
to meet the promotional cutoff in both May and Au- 
gust of 1997 (see Figure 3-3). 
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Figure 3-3 



Pefceintage of High and Moderate Risk 
African-American and Latino Students Who 
Did Not Meet the Cutoff and Were Retained 
Third Graders, 1997 




below GEs GEs below GEs GEs 

below below below below 

How far below second grade cutoff, based on test history 
□ African-American □ Latino 



How Did Ethnic Differences Impact Sixth Graders under the Policy? 

Main finding: The proportions of African-American and Latino sixth graders 
who were at risk under the policy were quite similar. In additiony there was 
little dijference in passing rates within risk categories. African-American sixth 
graders werey howevery much more likely to be retainedy largely because in 1997 
many Latino students were promoted despite having test scores below the cutoff. 

Latino students were again excluded at higher rates in the sixth grade. Ap- 
proximately 23 percent of Latino sixth graders versus only 16 percent of 
African-American sixth graders were excluded from the promotional policy. 
Within risk categories, there was very little difference between the percent- 
age of African-American and Latino students who failed to meet the pro- 







motional test cutoff. Importantly, 
however, African-American sixth 
graders were more likely to be re- 
tained, largely because of differ- 
ences in promotion rates among 
those who did not meet the cut- 
off. For example, 73 percent of 
African-American and 74 percent 
of Latino sixth graders who had 
to make up 1.5 or more Grade 
Equivalents (GEs) to pass the cut- 
off failed to reach a 5.3 by Au- 
gust of 1997 (see Figure 3-4). Yet 
55 percent of African-American 
students in this category versus 38 
percent of Latinos were retained. 
This translates into a promotion 
rate among students who did not 
meet the cutoff of 25 percent for 
high risk African-American stu- 
dents versus 49 percent for Latinos. 

One explanation for this result 
is that educators may have been 
particularly sensitive to the fact 
that students whose test scores 
were being counted for the first 
time — those who were entering 
their fourth year in bilingual edu- 
cation — might not do particu- 
larly well. We found, however, 
that Latino sixth graders were 
more likely to be promoted de- 
spite having test scores below the 
cutoff regardless of how long they 
had been in the system. If we look 
at the schools that received the 
most waivers in 1997, the major- 
ity were concentrated in neigh- 
borhoods that had high 
concentrations of Latino students 
(see Figure 3-5 on page 46). 
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How Did Ethnic Differences 
Impact Eighth Graders under 
the Policy? 

Main finding: There was little dif- 
ference in the experience of Latino 
and African-American eighth grad- 
ers. Retention rates were quite simi- 
lar in eighth grade. 

We noted earlier that exclusion 
rates declined significantly in later 
grades because many more bilin- 
gual students were included un- 
der the policy, and the reason for 
exclusion shifted to special edu- 
cation. As a result, by eighth 
grade, there was little difference 
in the exclusion rate between Af- 
rican-American and Latino stu- 
dents. For students included in 
the policy, passing rates were quite 
comparable between the two 
groups (see Figure 3-6 on page 
47). High risk Latino eighth grad- 
ers were slightly more likely to be 
promoted despite not meeting 
the test cutoff in 1997, but this 
difference had only a moderate 
impact on the overall retention 
rate. Approximately 39 percent of 
high risk Latino eighth graders 
who did not meet the promo- 
tional criteria in August 1997 
were promoted, versus 3 1 percent 
of high risk African-American 
eighth graders. In 1 997, 1 4 per- 
cent of African-American eighth 
graders and 12 percent of Latino 
eighth graders were retained or 
sent to Transition Centers (see 
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Figure 3-4 



Percentage of High and Moderate Risk 
African-American and Latino Students Who 
Did Not Meet the Cutoff and Were Retained 




below GEs GEs below GEs GEs 

below below below below 

How far below fifth grade cutoff, based on test history 
□ African-American □ Latino 



Figure 3-7 on page 48). Thus, the major difference between African- 
American and Latino students in both risk and performance occurred 
in the third grade. 

Were There Gender Differences? 

Main finding: Across all grades, there were only small differences in the ad- 
ministrative treatment or in the performance of boys and girls under the pro- 
motional policy. Boys were retained at higher rates because they were, in general, 
farther behind on entry into those grades. In both the third and sixth grades, 
many more boys than girls faced high to moderate risk of not meeting the cutoff. 
Within risk categories, however, passing and retention rates were similar. 
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Figure 3-5 



Schools Promoting the Most 6th Graders below the Cutoff 
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Previous research on grade reten- 
tion finds that boys are much 
more likely to be retained, par- 
ticularly in the early grades.^* 
Poor school performance and 
high dropout rates among minor- 
ity males is a national concern. 
How have boys and girls fared 
under this policy? 

Across all grades, boys were 
slightly more likely to be retained, 
with the largest difference occur- 
ring in the third grade. In third 
grade, 24 percent of boys versus 
19 percent of girls were retained 
(see Figure 3-8 on page 48). There 
is a straightforward explanation 
for why this occurred: boys were 
farther behind upon entry into 
third grade and, therefore, faced 
a higher risk of not meeting the 
cutoff. Using a statistical estimate 
of how far behind the test-score 
cutoff students were upon entry 
into the promotional gate grade, 
55 percent of third grade boys 
versus 50 percent of third grade 
girls faced moderate to high risk 
of not meeting the promotional 
criteria (see Figure 3-9 on page 
49 for proportions in each cat- 
egory). Boys were also more likely 
to be at risk in the sixth and 
eighth grades. 

Among students who faced 
similar test-score deficits, there 
were only moderate gender dif- 
ferences in the proportion of boys 
and girls who failed to meet the 
promotional criteria and who 
were retained. Thus, the issue of 




Figure 3-6 



Percentage of High and tVioderate IRisk 
Afirican-Amencan and Latino Students Who 
Did Not Meet the Cutoff and Were Retained 
Eighth Graders, 1997 




i LI i U ! U i U t 

>1 .5 GEs 1 .5 to 1 1 to .5 >1 .5 GEs 1 .5 to 1 1 to .5 

below GEs GEs below GEs GEs 



below below below below 

How far below seventh grade cutoff, based on test history 

□ African-American □ Latino 



gender appears to be a broader issue of the poorer school performance of 
boys, particularly in the earlier grades, rather than an issue of differences in 
performance linked to the policy. 

What Have We Learned? 

T he statistics presented in this section underscore the complexity of what 
it means to end social promotion in large urban school systems where 
an increasing number of students are immigrants who may be learning 
English while trying to gain the higher levels of basic skills now expected of 
all students. Chicago made the decision that students who were in bilin- 
gual education for fewer than three years would be exempted from the 
promotional policy. In 1999, this exclusion criterion was raised to four 
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Figure 3-7 

Percent of African-American and Latino 
Students Retained 




3rd grade 6th grade 8th grade 



□ African-American 



□ Latino 



Figure 3-8 

Percent of Students Retained by Gender 

1997 



Boys n Girls 




years. This means that the major- 
ity of Latino third graders and a 
high proportion of Latino sixth 
graders are excluded from the 
policy. This is an important 
trend given that Latinos are the 
fastest growing group of stu- 
dents in the Chicago Public 
Schools. It is beyond the scope 
of this report to address the ques- 
tion of whether excluding Latino 
students is the most education- 
ally appropriate policy decision. 
Clearly, this policy gets at the 
heart of the ongoing debate over 
the rate at which language-minor- 
ity students should move into En- 
glish-speaking classrooms, and the 
rate at which those students should 
be expected to meet similar criteria 
to non-language-minority students 
for school performance. 

What is clear is that the exclu- 
sion of Latino students in the 
early grades means schools with 
Latino student populations face 
a very different problem in the 
sixth and eighth grades than other 
schools. One argument for end- 
ing social promotion at the third 
grade is that early intervention 
will spur students to reach mini- 
mum standards and reduce their 
risk later on. But, as the policy 
currently stands, many Latino 
students in the Chicago Public 
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Figure 3-9 

Boys and Girls at Risk for Not Meeting the Cutoff 

Third, Sixth, and Eighth Graders, 1997 




High risk Moderate risk 



Note: High risk students are those whose predicted test scores indicate they would need to make up over 1 .5 GEs 
in a year to make the test score cutoff. Moderate risk students are those who would have to make up .5 to 1 .5 GEs in a 
year to make the cutoff. 



Schools will not face these cutoffs until the sixth or 
even eighth grade. First, schools with Latino student 
populations should consider using the knowledge that 
students will eventually face the cutoffs and extend 
early supportive services to those who are excluded in 
the early grades. Second, Latino sixth and eighth grade 
teachers clearly face a more difficult problem because 
they must decide whether student deficits reflect a lack 
of language proficiency or another undiagnosed learn- 
ing problem. Thus, schools with Latino student popu- 
lations may need to direct extra resources and supports 
to these grades. 

One of our reasons for looking at ethnic/ racial dif- 
ferences in performance under the policy was to ad- 



dress the concern that Latino students, particularly 
more recent immigrants, would face a greater risk of 
retention. We find quite the opposite. First, among 
those students included under the policy. Latino stu- 
dents performed as well as, or, in the third grade, bet- 
ter than African-American students. And, second, we 
find that the implementation of the policy in Chicago 
appeared to be particularly sensitive to the impact of 
language proficiency on student performance. Most 
Latino third graders were excluded from the promo- 
tion criteria. In addition, a higher proportion of Latino 
than African-American students were promoted de- 
spite having test scores below the promotional criteria 
in August of 1997. 
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hicagos effort to end social promotion is an integrated set of initia- 
tives designed to focus attention on lower performing students and 
raise their test scores to minimum promotional cutoffs — during the 
year before testing, over the summer, and for those who are retained, dur- 
ing the next year. By using minimum test scores as the criteria for promo- 
tion and by using the threat of retention if these criteria are not met, the 
initiative seeks to increase student effort and focus program resources and 
teacher attention on improving basic skills. Those third, sixth, and eighth 
graders who fail to meet the test cutoffs in May are given a second chance 
to meet the test criteria after more intensive instructional time in reading 
and mathematics in the Summer Bridge program. Most of those who fail 
again in August are retained. The initiative uses the retention year, com- 
bined with a second round of program supports, to try to redress contin- 
ued poor performance. 

This report has focused on three broad areas. First, it described the imple- 
mentation of the policy during the first two years, examining the flows of 
students through the policy during 1997 and 1998. Second, it compared 
ITBS achievement trends for students affected by the policy to those of 
students before the policy was implemented and looked at trends in stu- 
dents’ test performance in the year before and after promotion or reten- 
tion. And, third, it examined ethnic and gender differences in the effects 
of the policy. This section is organized to summarize the main findings of 
the report and to highlight the questions they raise for policymakers both 
locally and nationally. 

Testing the Theory of Action in Chicago: Were More Students Meet- 
ing the Test Cutoffs? The premise of the CPS policy for ending social 
promotion is that setting minimum test-score standards for promotion 
and providing extra instructional time to students during the school year 
and summer will allow more students to meet the minimum test cutoffs 
for their grade. We began to evaluate this claim by comparing the perfor- 
mance of students who were subject to the policy in 1997 and 1998 with 
that of a previous group of CPS students (third, sixth, and eighth graders 
in 1995) who were not subject to the promotional criteria. 
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There have been impressive increases in the pro- 
portion of students who meet minimum test-score 
cutoffs for promotion. Overall, many more students 
had ITBS scores that met the minimum cutoff required 
for promotion in 1997 and 1998 than did before the 
policy. In this respect, the CPS policy looks like a suc- 
cess. Increases in the sixth and eighth grades reflected 
both improvements in student performance during the 
school year and the effect of Summer Bridge. The pro- 
portion of all sixth graders who reached the minimum 
promotional cutoff was 20 percent higher in 1997 than 



[Perhaps] the most positive finding in 
this report is that across all three grades 
the Summer Bridge program ... has 
been one of the most successful aspects 
of the policy . . . 



in 1995 and 21 percent higher among eighth graders. 
In the second year of the policy, passing rates among 
sixth graders were even better. 

Third grade is an area of concern. There was little 
increase in the proportion of third graders who met 
the minimum cutoff in reading from May 1995 to 
May 1997. The performance of third graders during 
the school year was better in 1998 than in 1997, but 
these improvements were largely concentrated among 
those students who had skill levels already close to the 
cutoff. More third graders eventually reached the pro- 
motional cutoff because of increases in their test scores 
after Summer Bridge. Indeed, perhaps the most posi- 
tive finding in this report is that across all three grades 
the Summer Bridge program — and the second chance 
it affords — has been one of the most successful aspects 
of the policy, accounting for a large proportion of the 
improvements in passing rates. 



O 




The performance of students with low skills 
showed the greatest improvement. Most impressive 
is that increases in passing rates were greatest among 
students with the lowest skills. We called students high 
risk if they needed to increase their ITBS scores by 1 .5 
GEs or more in a year in order to meet the promo- 
tional test cutoff and moderate risk if they needed “av- 
erage” to “above average” test score gains in the 
promotional testing year (.5 to 1 .5 GEs). We used this 
wide category for moderate risk because there is so 
much test score fluctuation from year to year. In 1995, 
about half of students who had test scores from the 
previous year already close to the cutoff (about .5 GEs 
below) actually didht meet the test score cutoff by the 
end of the school year. 

Many CPS students fell into one of these risk cat- 
egories. Almost half of third graders and almost 40 
percent of sixth and eighth graders had such low read- 
ing scores that they could be considered at risk of fail- 
ing to meet the promotional cutoffs. Among high risk 
students, the proportion who were able to meet the 
test cutoff increased from 4 to 34 percent among sixth 
graders and from 12 to 49 percent among eighth grad- 
ers between 1995 and 1997. 

The picture is mixed on whether getting students 
up to a test-score cutoff in one year allows them to 
do better the next year. The argument for getting 
more students to meet a minimum test score cutoff is 
that this lays the basis for long-term school success, 
while promoting them without basic skills places them 
in a position of falling farther behind. We took a pre- 
liminary look at this claim by comparing test score 
trends in the year before the promotional grade, over 
the summer, and in the year after retention and pro- 
motion for students in 1995 (pre-policy) and 1997 
(first-year policy). The evidence here is quite mixed. 
The good news is that the larger proportion of stu- 
dents who made the cutoff in May 1997 had two-year 
test score gains that were comparable to the smaller 
proportion of students in 1995 who would have met 
the promotional cutoff had it been in place. This sug- 
gests that an increased number of students are now on 
track under the policy. 
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At the same time, however, large test score in- 
creases in Summer Bridge were not followed by im- 
proved performance the next year. While Summer 
Bridge raised students’ performance briefly, there is 
no evidence that it altered the overall pattern of school- 
year achievement for these students. After Summer 
Bridge, students reverted to learning about as much 
the next year as they did previously. There are two 
explanations for this trend. First, it suggests that the 
increases we are seeing in Summer Bridge may reflect 
testing effects versus learning gains. The National Re- 
search Council raised this issue in their report on high 
stakes testing when they argued: 

... the available data provide no means of distin- 
guishing true increases in student learning from 
artifactual gains. Such gains would be expected from 
the combined effect of teaching to the test, repeated 
use of a similar test, and in the case of Summer 
Bridge program, the initial selection of students with 
low scores on the test?^ 

While we can’t speak to the first two of these criti- 
cisms, our forthcoming technical report on the Sum- 
mer Bridge program will present a more rigorous 
analysis of test score increases that correct for these 
selection effects. This report finds that the estimated 
gains in test scores in Summer Bridge are slightly over- 
estimated by simple descriptive data, but remain large 
and significant when more rigorous statistical meth- 
ods are used. 

A second interpretation of Summer Bridge test score 
gains may be that raising skills during the summer does 
not carry over to the next school year as much as has 
been anticipated. As a result. Summer Bridge, at least 
in the short run, allowed students to stay on track by 
compensating for poor school-year gains with large 
summer gains. Trends in test scores among students 
promoted after Summer Bridge are certainly positive 
when compared to the experience of students who were 
previously socially promoted. Nonetheless, these stu- 
dents’ weak school-year gains relative to summer meant 
that they remained at risk for retention the next time 



they faced the test cutoff. This second' interpretation 
suggests that positive summer experiences cannot com- 
pensate for weak school-year instruction and that these 
students need improved instruction across their school 
careers. It may also suggest that students in Summer 
Bridge may be a particularly vulnerable group of stu- 
dents who might well need sustained attention across 
their school careers. 

What Do We Know about the 
Progress of Retained Students? 

T his report also addressed the most controversial as 
pect of the promotional policy: the decision to re- 
tain students. Even if the policy produces benefits for 
students who are promoted, the continued feasibility 
of this initiative will ultimately depend on whether 
CPS can effectively address poor performance among 
students who are retained. Results from the first group 
of retained students are far from sanguine. Only one- 
fourth of retained eighth graders and one-third of re- 
tained third and sixth graders in 1997 made “normal” 
progress during the following school year, meaning that 
they stayed in the school system, were again subject to 
the policy, and passed the test cutoff the next May. 

Thus, retained students did not do better than pre- 
viously socially promoted students. The progress 
among retained third graders was most troubling. Over 
the two years between the end of second grade and 
the end of the second time through third grade, the 
average ITBS reading scores of these students increased 
only 1.2 GEs compared to 1.5 GEs for students with 
similar test scores who had been promoted prior to 
the policy. 

Also troubling is that one-year dropout rates among 
eighth graders with low skills are higher under this 
policy. We will be continuing to chart the progress of 
this group during 1999, when those students will turn 
1 6 and face the time when dropping out normally be- 
gins to occur. 

How do we interpret the performance of re- 
tained students? In short, Chicago has not solved 
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the problem of poor performance among those who 
do not meet the minimum test cutoffs and are retained. 
Both the history of prior attempts to redress poor per- 
formance with retention and previous research would 
clearly have predicted this finding. Few studies of re- 
tention have found positive impacts, and most sug- 
gest that retained students do no better than socially 
promoted students. This is clearly the most difficult 
problem to address. The CPS policy now highlights a 
group of students who are facing significant barriers 
to learning and are falling farther and farther behind. 
These students are now identified as not meeting the 
promotional criteria and are retained, while in the past 
they were socially promoted. How best to advance the 
learning of students whose test scores are not improv- 
ing remains unclear. 

The administration clearly needs to take a close look 
at the adequacy of its current efforts in the retained 
year. CPS has continued to experiment with alterna- 
tives to retention and with directing resources to stu- 
dents in the second year. At present, students in the 
retained year are provided with substantial extra re- 
sources through Lighthouse, reduced class sizes, and 
extra instructional support in schools hit hard by re- 
tention. In our subsequent work, we will be looking 
specifically at how these various interventions in the 
retained year — Lighthouse, mid-year promotions, and 
placement of students in smaller classes — may shape 
students’ learning in comparison to students who are 
simply retained and placed in regular classrooms. The 
current strategy, however, of using Lighthouse, a sec- 
ond summer of Summer Bridge, and a second time 
through the policy, amounts to a double dose of the 
same medicine the students received the year before. 
We find that retained students showed little progress 
in the year after retention and during Summer Bridge. 
If incentives, extra instructional time, and a summer 
program did not work to improve these students’ 
skills the first time around, will a second year of the 
same program produce greater benefits? This 
policy — because it identifies students who don’t 
meet the test cutoff — provides an opportunity to 
clearly diagnose these students’ problems and provide 







more intensive interventions during the second year. 
Doing so, however, may require a different approach 
than that used in the first year. It may also require 
more sustained intervention in particular schools. 

How do we interpret the weaker effect of the 
policy in the third grade? Another key finding in 
this report is that the performance of third graders was 
significantly poorer under this policy than that of sixth 
and eighth graders. In some ways, this finding is sur- 
prising. The rhetoric of early intervention surmises that 
intervening in sixth and eighth grade would be too 
late to remediate poor skills. In retrospect, however, 
the initial design of this policy may be more appropri- 
ate for older students. 



In stiorl, Cliicap has iiol solved Itie 
problem of poor performance among 
those who do not meet ie minimom 
test cutoffs and are retained. 



The CPS initiative relies heavily on incentives for 
students to work harder and on producing large gains 
in short intensive periods such as summer school. In 
this respect, the finding about third graders might not 
be surprising. Eighth graders face the greatest costs in 
not meeting the test cutoff (they don’t go on to high 
school) and have the greatest capacity to shape their 
school performance through their own motivation and 
effort. It might also be true that eighth graders are at a 
time in their development when they can more easily 
learn in intensive periods of immersion. In contrast, 
third graders may be less sensitive to the threat of re- 
tention, less able to shape their own learning by ef- 
fort, and less likely to overcome barriers through 
intensive learning spurts.^^ 

Raising students’ skills in the early grades may re- 
quire a different approach. The CPS administrations 

.. 
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current strategy in addressing the poorest performers 
among third graders is to extend the program without 
retention into the earlier grades. Beginning in the 
1999-2000 school year, the Lighthouse and Summer 
Bridge programs have been extended to first and sec- 
ond grade students who are not at grade level. This 
means that the administration has extended the school 
day and school year for at risk students for three years 
prior to the test cutoff in third grade. 

Another and not mutually exclusive response to the 
poorer outcomes among third graders would be to 
focus more attention on improving the core instruc- 
tional capacity of teachers in the early grades. In gen- 
eral, CPS has used a strategy of supplementing 
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instruction by increasing instructional time in after- 
school programs and during the summer. CPS has also 
tried to infuse the instructional strategy used in these 
programs (the centrally developed curriculum) into the 
school year. Thus the administration has worked to 
raise test scores among low-performing students with- 
out having to address questions regarding the adequacy 
of instruction during the school day or spend resources 
to increase teachers’ capacity to teach and to meet stu- 
dents’ needs more successfully. 

Taken together, one interpretation of the findings 
of this report is that the CPS social promotion policy 
has worked to reveal a core problem — the adequacy 
of instruction during the school year. If this is indeed 
a problem, then the ultimate success of this policy will 
depend upon whether the extra program efforts and 
extra efforts on the part of students are matched with 



an increase in the capacity of teachers to build early 
literacy and numeracy and to diagnose and address 
students’ problems when they are not progressing. 



Rndings of the Report in 
e Chicago Approach to 



ne of the purposes of this report was to set out for 
a national audience the various components of the 
Chicago policy and its implementation process dur- 
ing its first two years. As we noted in the Introduc- 
tion, all eyes are on Chicago in this regard. CPS has 
embarked on a rigorous attempt to raise standards by 
focusing on individual student performance. This 
policy is often described as a “get tough” approach, 
but our look at the design and implementation of the 
policy finds that there are two aspects of Chicago’s ef- 
forts often overlooked in the national debate. First, 
the initial design of the policy reflected a concern that 
students who are bilingual and those who have special 
education needs should not be held to the same strict 
standards as other students in their grade. As a result, 
almost one- third of third graders were initially excluded 
from the promotion policy. In the first two years, the 
administration also liberally used waivers, particularly 
among Latino students, both prior to the retention/ 
promotion decision and in assessing the progress of 
retained students the second time through this policy. 
The use of waivers, or of promoting students who did 
not meet the minimum test score cutoff, allowed Chi- 
cago to substantially reduce the proportion of students 
who were retained. Without such flexibility in the 
Chicago policy, its initial impact would have been 
much more disconcerting. 

We do not mean to suggest that Chicago should 
end waivers or have all students included under the 
policy. Rather, we argue that the simplistic “sound bites” 
and rhetoric often used by those on both sides of the 
debate to characterize Chicago’s efforts are misleading 
and ultimately dysfunctional. Such over-simplification 
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This report is the first in a series of investigations from a 
multi-year study of Chicago’s promotion policy. The study 
brings together analysis of achievement and high school 
transcript data; ongoing surveys of teachers and students 
conducted in 1994, 1997, and 1999; and a qualitative study 
of the experience of 100 students, their families, and teach- 
ers under the policy during the 1998-1999 and 1999-2000 
school years. 

Our intent in this first report is simply to describe the 
general landscape of an important reform effort. As we ex- 
pected, these initial findings raise many more questions than 
they answer. In subsequent reports we will probe issues such 
as effects of the policy on long-term student achievement 
trends and high school completion rates, on the changing 
nature of classroom instruction, on parental involvement, 
and on students’ own attitudes. We will conduct a cost study 
to evaluate the benefits of this approach versus other alter- 
native methods of improving student performance in both 
the early and middle school years. We also expect that pub- 
lic discussions following the release of the report will sharpen 
our current research agenda on this topic and likely reshape 
it in important ways as well. 

Subsequent reports will take several forms. First, we are 
already at work on three technical briefs that offer more 
detailed statistical analyses of the nature of achievement 
gains associated with the policy. Second, future yearly fol- 
low-up reports will present updated information on stu- 
dents’ flows through the policy and will track the learning 
trajectories of students over time. And, third, subsequent 
public reports will focus on the following issues: 

A Closer Look at Summer Bridge. The large increases in 
test scores associated with Summer Bridge suggest we take 
a closer look at this program. Several key features may be 
contributing to these increases: substantially reduced class 
sizes, the selective recruitment of teachers for the program, 
a standardized curriculum aligned to the ITBS, and more 
intensive and effective use of instructional time. We will be 
reporting on the characteristics of teachers who teach in 
the program versus those in the regular school year and will 
be looking at how students view their experience in the 
summer. Finally, we will be looking more carefully at the 
characteristics of students who made and did not make sub- 
stantial learning gains during the summer and at character- 
istics that allowed some Summer Bridge sites to be more 
effective than others. 



This Summer Bridge report, along with our technical 
report on Summer Bridge learning gains, will focus on prob- 
lems in interpreting test score increases in summer programs 
when there are potential important testing effects that might 
lead to overestimates of program impacts. 

Looking inside the Box: Effects on Instruction and Pro- 
fessional Practice. A key question surrounding CPS’s pro- 
motional initiative is its effects on classroom instruction. 
Many educators worry that the policy encourages emphasis 
on test preparation and basic skills and diminishes atten- 
tion to other subject matter and more complex academic 
skills. On the other hand, by design the policy is intended 
to redirect instructional resources to accelerate the progress 
of students who need help and, by reducing the spread of 
achievement in post- promotional grades, allow teachers to 
intensify the pace of subsequent instruction. In this report, 
we will use longitudinal survey data on teachers’ reports of 
content coverage, pedagogical practices, instructional pac- 
ing, and time spent on test preparation to examine how the 
policy may be shaping instructional practices. 

A Closer Look at Retention. We found in this report that 
students who were retained under the policy had very poor 
learning gains in the year before promotion, during the sum- 
mer, and in the retention year. Many had a hard time reach- 
ing the test score standard even after an additional school 
year and summer of instruction. These findings call for a 
more intensive investigation of this aspect of the program. 
We will look more closely at the educational histories of 
these students to see if there are any clues which suggest 
how schools might be able to intervene earlier and more 
effectively. We will also look more closely at how schools 
are programming for these children. Preliminary analysis 
suggests considerable variability among schools in how much 
students are gaining on the ITBS during their retained year. 

* * ♦ ♦ ♦ 

In the spirit of opening the public discussion about this 
most important CPS policy, we welcome your reactions to 
these ideas and specifically invite your suggestions on where 
we should go next. We will be maintaining a running 
commentary on our web site, www.consortium- 
chicago.org, about this report and our future research. 
We welcome your comments. 
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encourages critics to ignore the complex evolving 
nature of the policy and the serious commitment it 
signals to raising student achievement and providing 
extra supports for students at every stage of the learn- 
ing process. It also encourages other districts to truly 
“end social promotion” without any of the safety nets, 
supplemental education resources, and attention to 
refining the policy at work in Chicago. The Chicago 
experience demonstrates the realities of urban educa- 
tion, where high rates of immigration and the com- 
plexities of childrens lives substantially complicate the 
idea of setting test cutoffs and then easily applying 
them. National efforts spurred on by the politically 
appealing rhetoric of “get tough” policies will be mis- 
guided if they do not also pay attention to the impor- 
tance of flexibility at the local level, particularly as 
policies interact with sensitive terrains like bilingual 
education. 

Second, we emphasized that early experience with 
the Chicago policy drives home the importance of care- 
fully setting test scores for promotion, paying atten- 
tion at each step along the way to which students the 
policy applies to, attending to the needs of those stu- 
dents who do not meet cutoffs, and committing re- 
sources to programmatic initiatives. One point stands 
out clearly: The CPS is committing enormous fiscal 
resources to this initiative in the form of extended day 
programs, summer school, and extra years of school 
with reduced class sizes. While a full accounting of 
the specific consequences of each of these initiatives 
will take several more years to accrue, even at this rela- 
tively early juncture one observation for other districts 
is clear. Do not attempt to implement this policy un- 
less your school district is willing to invest, as Chicago 
has, substantial fiscal and administrative resources. 

How is this policy changing? From the perspec- 
tive of CPS leadership, an important and often over- 
looked characteristic of the Chicago effort is that the 
policy is intended to evolve over time. This has im- 
portant implications for our research because we are 
studying a moving target. CPS is making continued 
efforts to fine-tune the policy. Many of the issues docu- 



mented in this report have already been recognized 
and in some cases new directions have been taken. For 
example, in the year 2000 all CPS first and second 
graders who have ITBS scores below grade level will 
be required to participate in Summer Bridge in an ef- 
fort to address the relatively poor effects of the policy 
in the third grade. In addition, the CPS continues to 
raise the test-score cutoff. In 1999-2000, the test 
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scores required for promotion will be raised in all 
three gate grades. 

Perhaps the most important change in the policy is 
the administrations plan to move beyond simple test- 
score cutoffs for promotion to more inclusive criteria 
that will include grades, attendance, and learning gains 
during the school year. In this way, the administration 
hopes to allow students to garner the benefit of pro- 
motional cutoffs while simultaneously rewarding stu- 
dents for effort and demonstrated performance and 
correcting the deficits of a policy that relies on a 
narrow and crude indicator of performance such as 
the ITBS. 

This is an approach that would clearly be supported 
by many of the critics of the policy as well as testing 
experts who caution strongly against sole reliance on 
ITBS Grade Equivalents to make promotional deci- 
sions (see Sidebar 4).^® The primary reliance on a single 
ITBS score, coupled with waivers, was expedient in 
initiating the policy, but may not continue to serve 
the system well. The use of the term “waiver” intro- 
duces a sense of arbitrariness and serves to undermine 
what appears to be the goal of minimizing retention 
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while using more inclusive criteria for promotion. A 
reformulating of the promotional criteria may be in 
order. In particular, CPS might consider uncoupling 
the criteria for participation in programs with the cri- 
teria for promotion or retention. The use of a single 
test score for participation in Summer Bridge, Light- 
house, and intervention programs is administratively 
easy to implement. But, it is clearly time for the ad- 
ministration to move forward with a more systematic 
formula for the promotional decision that formally al- 
lows for students’ grades, attendance, and learning 
growth. In this study, we have also shown how the 
accuracy of test scores can be improved by using stu- 
dents’ previous test score histories. All of these indices 
could be formulated into a standard, more accurate 



and more defensible promotional policy that con- 
tinues to send a strong message to students, par- 
ents, and teachers about the importance of effort 
and achievement. 

A more systematic formula for promotion would 
also allow the policy to be implemented in a way that 
clearly communicates goals to teachers and schools and 
ensures that all students who might be eligible for pro- 
motion under more inclusive criteria are promoted. 
Our look at racial and ethnic differences in the first 
year of the policy suggests that without such a con- 
certed and standardized approach, questions of eq- 
uity regarding waivers and retention may become a 
significant concern. 
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Concluding Comments 

T his report provided a first look at the implementa- 
tion of Chicago’s efforts to end social promotion. 
We relied largely on descriptive analysis in order to 
make the findings as accessible as possible to a broad 
audience. We are already engaged in more systematic 
analysis of each of the main findings in order to derive 
more precise answers to questions such as: What ex- 
actly is the effect of the policy on learning gains in the 
year before testing? What do we know about learning 
gains in the Summer Bridge program? And, to what 
extent are retained students’ learning trajectories dif- 
ferent from those of students who were socially pro- 
moted in the years before the policy? 

Our findings highlight the central tension that any 
school system will face in trying to raise achievement 
among low-performing students by using the threat 
of retention as a motivating factor. On the one hand, 
more students are now meeting a minimum test crite- 
ria for promotion. On the other hand, we find very 
troubling trends in the performance of retained stu- 
dents. And while Summer Bridge substantially helps 
many students, it does not appear to be enough. 

In the end, the verdict is out on whether Chicago’s 
initiatives are producing substantial benefits for stu- 
dents. Many of the main trends presented in this re- 
port will become clearer with another year of data 
collection. This additional year’s worth of data will al- 
low us to get a better look at the test score trends of 
students promoted after Summer Bridge, of retained 
students, and of students who initially met the test 
cutoffs in May. Time will also allow us to determine 



whether schools are able to sustain efforts and whether 
problems can be identified early on and more effec- 
tively addressed. For this reason, a number of follow- 
up reports will be forthcoming. It is still quite early in 
this evaluation to make statements about whether the 
policy is working. 

But policy time and research time are often not on 
the same clock. This report is intended to provide in- 
formation that documents potential positive effects and 
identifies problems and issues with which educators 
and policymakers still need to grapple without pro- 
viding definitive statements about their effects. Over 
the long term, the substantial expense of this policy 
raises this question of opportunity costs: Is this the 
best use of scarce resources to improve student learn- 
ing? And, do the benefits of the policy (more students 
meeting minimum standards) come at a cost, and if 
so, for whom? Many opponents of the policy worry 
that the narrow focus on raising ITBS scores in read- 
ing and math will substantially limit opportunities to 
learn in the Chicago Public Schools. The nature of 
the instructional effects of this policy will be one of 
the most important areas to assess in order to truly 
understand the positive and negative impacts of such 
high-stakes testing approaches to raising standards. A 
cost-analysis and a look at instruction effects will be 
the topic of subsequent reports. 

In closing, we expect that this report will raise more 
questions than we have actually answered here. This is 
not unusual during the first stage of an important 
research program. We hope that public debate 
around these findings will shape and sharpen the 
research that follows. 
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* It is important to recognize, however, that there is no agreed upon definition of the 
term “social promotion.” In this report, we define the term quite literally. Social pro- 
motion is making the decision to promote a student on the basis of their social devel- 
opment. The implication of the term social promotion is that students are being 
promoted without reaching minimum standards for their grade. 

^ Hauser (1 999). The National Research Council was organized by the National Acad- 
emy of Sciences in 1916. 

^ Hoover, Hieronymus, Frisbie, and Dunbar (1996). 

Heubert and Hauser (1999). 

^ In 1997, Transition Center students were allowed to retake the Iowa Tests of Basic 
Skills (ITBS) in January. In 1 998, this third chance was expanded to all grades. Stu- 
dents who pass the test cutoff at the end of the first semester are promoted at the 
beginning of the second semester with a required double (six hours per day) summer 
school at the end of the year. 

Hauser ( 1 999) , Heubert and Hauser ( 1 999) , Holmes ( 1 989) , Holmes and Matthews 
(1984), House (1998), Jackson (1975), and Shepard, Smith, and Marion (1996). 

^ Barro and Kolstad (1987); Byrnes (1989); Gottfredson, Fink, and Graham (1994); 
Grissom and Shepard (1989); Hauser (1999); Heubert and Hauser (1999); Hess and 
Lauber (1985); Holmes and Matthews (1984); House (1998); New York City Board 
of Education, Office of Educational Assessment (1988); Plummer and Graziano (1987); 
Roderick (1994). 

^Tomchin and Impara (1992). 

^ House (1998) and New York City Board of Education (1988). 

Hauser (1999). 

Fusaro (1997), Levin and Tsang (1987), and Smith (1998). 

Entwisle, Alexander, and Olson (1997); Heyns (1987). 



65 



62 



CHARTING REFORM IN CHICAGO 



In 1998, the promotional test cutofF for eighth graders was 
raised from 7.0 to 7.2, and then to 7.4 in 1999. In the year 
2000, the minimum cutoffs will be raised for both sixth and 
eighth graders. 

The promotional decisions of special education students are 
based on individual promotion plans that may include test 
scores as part of the criteria for promotion. We call students 
included if their promotional decisions were strictly based on 
the test-score cutoff. From 1996 through 1998, students were 
excluded if they were enrolled in a bilingual education pro- 
gram and if they had been enrolled for less than three com- 
plete years as of the prior September 30. In 1 999, that criteria 
was changed to four years. 

All students referred to in this section of the report are stu- 
dents who were included under the policy. 

These retention rates are for the proportion of included stu- 
dents retained under the policy. 

^^The number of retained students (10,1 19) and the number 
of promoted students (5, 793) do not add up to the total num- 
ber of students eligible for retention (16,744) because 832 stu- 
dents who were eligible for retention left the school system 
before promotion or retention. 

Guidelines for Promotion in the Chicago Public Schoob, p. 17. 

In this report, we calculated waiver rates as the proportion of 
students who were promoted despite failing to meet the pro- 
motional criteria by August. For example, in 1997, 7,650 third 
graders who were included under the policy did not meet pro- 
motional criteria in both subjects. Of these students, 4,796 
were retained, 2,497 were promoted, and 357 left the school 
system. Thus, for third graders, the waiver rate excluding stu- 
dents who left the system is 34 percent. 

We restricted the waiver calculation only to those who did 
not leave the school system. Among sixth graders, for example, 
2,031 received waivers, and 3,035 were retained (see Figure 1- 
3). Thus, the waiver rate can be calculated as the percent waived 
out of the 5, 066 eligible for retention, in this case 40 percent. 

This exclusion rate during the 1998 year is calculated with- 
out those students who left the school system. 



There is a fear that schools will use special education place- 
ments to exempt students from the policy. On the other hand, 
these placements may also signify that the policy is finally lead- 
ing teachers to pay closer attention and better diagnose stu- 
dents’ problems, resulting in much needed referrals. While this 
does not resolve the debate over whether more diagnoses 
should be happening, clearly these exclusion rates are not evi- 
dence of widespread use of placing retained students in spe- 
cial education. 

In this section we calculate passing rates on the basis of the 
progress of all retained students, including those who were 
excluded from the policy. We do this because exclusion in 
the second year is part of the outcome for students who 
are retained. 

In this section of the report, passing, waiver, and retention 
rates for 1997 refer to the 1996-1997 school year, and rates 
for 1998 refer to the 1997-1998 school year. Since passing can 
occur during May or August, and since waivers and retentions 
occur during August only, the end-of-school-year dates were 
used to simplify the time line. 

The CPS administration has argued that waivers should be- 
come more prevalent over time as the bar is raised and as other 
indicators of performance, such as grades, become more reli- 
able indicators of school performance. In 2000 when the bar 
is raised in all three grades, the administration has stated that 
students will be given extra leniency in passing these higher 
bars because grades, attendance, and learning growth over the 
school year will be given more weight in the promotional de- 
cision. This policy, however, has not yet been formalized or 
implemented. (“It’s Time to Evolve: Paul Valias Says as He 
Finishes Four Years.” Catalyst: Voices of Chicago School Reform. 
June 1999.) 

During the 1994-1995 school year, the year before the policy 
was put in place, the average third grader increased his or her 
reading and math ITBS scores by approximately seven months 
between the end of second grade and the end of third grade. 
The average testing gains for sixth graders were eight months 
in reading and 1.06 years in math. Testing gains for eighth 
graders were 1.1 years in reading and 1.6 years in math. 
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Given the testing gains outlined in Endnote 1 , it might seem 
confusing to consider students at risk if they need only one- 
half to one year test-score gains in the third, sixth or eighth 
grade. As seen in Figure 2-1, however, in 1995 only slightly 
more than half of these students actually met the promo- 
tional criteria. This reflects year-to-year fluctuations in 
ITBS scores that result in many who are close to the cutoff 
remaining at risk. 

A subsequent technical report will present a more rigorous 
statistical model to estimate the addition to test-score gains 
associated with the policy in the year before testing. This model 
uses data from 1990-1997 students. Thus, we compare testing 
gains in the third, sixth, and eighth grades to those observed 
in several groups of students in these grades prior to the policy 
rather than in just one year, 1995. This model also allows 
us to adjust for trends in achievement test scores across all 
grades in the school system. In general, the results of this 
more rigorous analysis are consistent with the descriptive 
results presented here. 

Prior work of the Consortium found that student testing 
gains began increasing in the Chicago Public Schools in the 
early 1990s. Throughout this decade, test scores in mathemat- 
ics have been rising steadily, where reading improvements have 
been more moderate. Test score increases over time, then, re- 
flect both a residual reform effect from the first wave of Chi- 
cago school reform and the effect of broader reform efforts 
instituted at the same time as the promotional policy such as 
the effect of putting schools on probation. 

^^This statistical model fits a linear growth curve to students 
testing trends in the years prior to the policy. For example, for 
sixth graders, we use students’ reading scores from first through 
fifth grade to obtain an estimate of students’ average testing 
gains across years. Using this model, we then predict a student’s 
fifth grade test score based on their average test-score growth 
in the years prior to the policy. Thus, if a student had an ab- 
normally low or high fifth grade test score, our model will 
correct for this. 

Again, the 1997 and 1995 designations used in this section 
refer to students who took the ITBS in May or August 1 997 
or in May 1995. These students began the school year in 1 996 
and 1994, respectively. 



As we saw in the previous section, most students who failed 
to meet the test cutoff in May participated in Summer Bridge 
(over 80 percent). In this graph, we restricted ourselves to these 
students so that we could look at test-score gains during Sum- 
mer Bridge compared to the school year. 

When looking at the dropout rates, we acknowledge one 
possibility that might be occurring— students who would have 
eventually dropped out may be dropping out earlier. That is, 
the overall dropout rate could remain constant even if the tim- 
ing of dropout changes. 

^"^The number of Latino students in the Chicago Public Schools 
has been increasing rapidly while the African-American stu- 
dent population has been declining. From the 1984-1985 to 
the 1996-1997 school year, the number of Latino children in 
the Chicago Public Schools increased by 43.5 percent or by 
nearly 41,000 students. 

Heubert and Hauser (1999), and Roderick (1995). 
^^Heubert and Hauser (1999), p. 132. 

Another interpretation of the differences in the effect across 
grades, however, might be that the policy is being applied to 
different groups of students across grades. Specifically, in the 
first section of this report, we found that the reasons that stu- 
dents were excluded from the policy differed across grades. In 
the third grade, most student were excluded because of bilin- 
gual education status, and by eighth grade most student were 
excluded because of special education status. This means that 
the high risk group in the third grade most likely contained 
students who would later be excluded from the policy. Thus, 
the policy could be having less of an effect on high risk third 
graders because students with the most severe difficulties, those 
who might later be placed in special education, were included 
in the third grade but excluded later on. This interpretation 
underscores the importance of moving away from the double 
dose approach in the retained year to spending more time, 
particularly in the early grades, closely diagnosing students’ 
learning problems. 

Bryk et al. (1998), Hoover et al. (1996), and Heubert and 
Hauser (1999). 
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First Time Third Graders 
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1 The excluded category includes students who 
took the test, but whose scores were excluded 
from reporting, and students who did not take the 
test. Of those students who took the test, the test 
scores of 4,619 students were excluded from re- 
porting. Of these, 2,323 were bilingual, 2,034 were 
special education, and 262 were both bilingual 
and special education students. An additional 
6,043 students did not take the test, perhaps be- 
cause of special education or bilingual status. 

2 2,587 of the 2,801 students who passed Sum- 
mer Bridge in 1 998 were promoted, 1 1 2 were re- 
tained, and 102 left the system. 



3 801 of the 4,790 students who failed Summer 
Bridge were promoted, 3,794 were retained, 193 
left the system, and 2 moved into non-graded 
special education. 

4 489 of the 1,717 students who did not take the 
test in Summer Bridge were promoted, 1 ,004 were 
retained, 219 left the system, and 5 moved into 
non-graded special education. 

5 The numbers in this column do not add up to 
exactly 34,295 because 53 students moved into 
non-graded special education between the two 
semesters. 
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Figure B 



Spring 1998 Test Results 

First Time Sixth Graders 
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1 The excluded category includes students who 
took the test but whose scores were excluded 
from reporting, and students who did not take the 
test. Of these students who took the test, the test 
scores of 4,048 students were excluded from re- 
porting. Of these, 362 were bilingual, 3,653 were 
special education, and 33 were both bilingual and 
special education students. An additional 1 ,693 
students did not take the test, perhaps because 
of special education or bilingual status. 

2 2,594 of 2,794 students who passed Summer 
Bridge in 1998 were promoted, 108 were re- 
tained, 90 left the system, and 2 entered 
Transition Centers. 



3 572 of the 2,789 students who failed Summer 
Bridge were promoted, 2,127 were retained, and 
90 left the system. 

4 537 of the 1 ,204 students who did not take the 
test in Summer Bridge were promoted, 506 were 
retained, 154 left the system, 6 moved into 
non-graded special education, and 1 entered 
a Transition Center. 

5 The numbers in this column do not add up to 
exactly 30,121 because 64 students moved into 
non-graded special education between the two 
semesters. 
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Figure C 



Spring 1998 Test Results 

First Time Eighth Graders 
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1 The excluded category includes students who 
took the test, but whose scores were excluded 
from reporting, and students who did not take the 
test. Of those students who took the test, the test 
scores of 4,033 students were excluded from re- 
porting. Of these, 276 were bilingual, 3,738 were 
special education, and 1 9 were both bilingual and 
special education students. An additional 1 ,664 
students did not take the test, perhaps because 
of special education or bilingual status. 

2 2,673 of the 2,883 students who passed Sum- 
mer Bridge in 1998 were promoted, 28 were re- 
tained, 181 left the system, and 1 entered a 
Transition Center. 



3 145 of the 2,322 students who failed Summer 
Bridge were promoted, 1 ,150 were retained, 170 
left the system, 1 moved into non-graded special 
education, and 856 entered Transition Centers. 

4 495 of the 1 ,128 students who did not take the 
test in Summer Bridge were promoted, 264 were 
retained, 242 left the system, 3 moved into non- 
graded special education, and 124 entered 
Transition Centers. 

5 The numbers in this column do not add up to 
exactly 28,687 because 1 32 students moved into 
non-graded special education between the two 
semesters. 
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